Configuring channels and the job runner¶
What is the job runner?¶
The job runner is the main process managing the dispatch of delayed jobs to available Odoo workers
How does it work?¶
It starts as a thread in the Odoo main process
It receives postgres NOTIFY messages each time jobs are added or updated in the queue_job table.
It maintains an in-memory priority queue of jobs that is populated from the queue_job tables in all databases.
It does not run jobs itself, but asks Odoo to run them through an anonymous
/queue_job/runjobHTTP request. 1
How to use it?¶
Optionally adjust your configuration through environment variables:
ODOO_QUEUE_JOB_CHANNELS=root:4(or any other channels configuration), default
http_portor 8069 if unset.
ODOO_QUEUE_JOB_HTTP_AUTH_USER=jobrunner, default empty.
ODOO_QUEUE_JOB_HTTP_AUTH_PASSWORD=s3cr3t, default empty.
Alternatively, configure the channels through the Odoo configuration file, like:
[queue_job] channels = root:4 scheme = https host = load-balancer port = 443 http_auth_user = jobrunner http_auth_password = s3cr3t jobrunner_db_host = master-db jobrunner_db_port = 5432
Or, if using
anybox.recipe.odoo, add this to your buildout configuration:
[odoo] recipe = anybox.recipe.odoo (...) queue_job.channels = root:4 queue_job.scheme = https queue_job.host = load-balancer queue_job.port = 443 queue_job.http_auth_user = jobrunner queue_job.http_auth_password = s3cr3t
Start Odoo with
--workersgreater than 1 2, or set the
server_wide_modulesoption in The Odoo configuration file:
[options] (...) workers = 4 server_wide_modules = web,web_kanban,queue_job (...)
Or, if using
[odoo] recipe = anybox.recipe.odoo (...) options.workers = 4 options.server_wide_modules = web,web_kanban,queue_job
Confirm the runner is starting correctly by checking the odoo log file:
...INFO...queue_job.jobrunner.runner: starting ...INFO...queue_job.jobrunner.runner: initializing database connections ...INFO...queue_job.jobrunner.runner: queue job runner ready for db <dbname> ...INFO...queue_job.jobrunner.runner: database connections ready
Create jobs (eg using base_import_async) and observe they start immediately and in parallel.
Tip: to enable debug logging for the queue job, use
After creating a new database or installing queue_job on an existing database, Odoo must be restarted for the runner to detect it.
When Odoo shuts down normally, it waits for running jobs to finish. However, when the Odoo server crashes or is otherwise force-stopped, running jobs are interrupted while the runner has no chance to know they have been aborted. In such situations, jobs may remain in
enqueuedstate after the Odoo server is halted. Since the runner has no way to know if they are actually running or not, and does not know for sure if it is safe to restart the jobs, it does not attempt to restart them automatically. Such stale jobs therefore fill the running queue and prevent other jobs to start. You must therefore requeue them manually, either from the Jobs view, or by running the following SQL statement before starting Odoo:
update queue_job set state='pending' where state in ('started', 'enqueued')
What is a channel?¶
Channel(name, parent, capacity=None, sequential=False, throttle=0)
A channel for jobs, with a maximum capacity.
When jobs are created by queue_job modules, they may be associated to a job channel. Jobs with no channel are inserted into the root channel.
Job channels are joined in a hierarchy down to the root channel. When a job channel has available capacity, jobs are dequeued, marked as running in the channel and are inserted into the queue of the parent channel where they wait for available capacity and so on.
Job channels can be visualized as water channels with a given flow limit (= capacity). Channels are joined together in a downstream channel and the flow limit of the downstream channel limits upstream channels.:
---------------------+ | | Ch. A C:4,Q:12,R:4 +----------------------- ---------------------+ Ch. root C:5,Q:0,R:4 | ---------------------+ Ch. B C:1,Q:0,R:0 ---------------------+-----------------------
The above diagram illustrates two channels joining in the root channel. The root channel has a capacity of 5, and 4 running jobs coming from Channel A. Channel A has a capacity of 4, all in use (passed down to the root channel), and 12 jobs enqueued. Channel B has a capacity of 1, none in use. This means that whenever a new job comes in channel B, there will be available room for it to run in the root channel.
Note that from the point of view of a channel, ‘running’ means enqueued in the downstream channel. Only jobs marked running in the root channel are actually sent to Odoo for execution.
Should a downstream channel have less capacity than its upstream channels, jobs going downstream will be enqueued in the downstream channel, and compete normally according to their properties (priority, etc).
Using this technique, it is possible to enforce sequence in a channel with a capacity of 1. It is also possible to dedicate a channel with a limited capacity for application-autocreated subchannels without risking to overflow the system.
How to configure Channels?¶
ODOO_QUEUE_JOB_CHANNELS environment variable must be
set before starting Odoo in order to enable the job runner
and configure the capacity of the channels.
The general syntax is
Intermediate subchannels which are not configured explicitly are autocreated with an unlimited capacity (except the root channel which if not configured gets a default capacity of 1).
A delay in seconds between jobs can be set at the channel level with
root:4: allow up to 4 concurrent jobs in the root channel.
root:4,root.sub:2: allow up to 4 concurrent jobs in the root channel and up to 2 concurrent jobs in the channel named
sub:2: the same.
root:4:throttle=2: wait at least 2 seconds before starting the next job