Job Runner

This is the API documentation for the job runner.

These classes are not intended for use by module developers.

What is the job runner?

This is an alternative to connector workers, with the goal of resolving issues due to the polling nature of workers:

• jobs do not start immediately even if there is a free connector worker,
• connector workers may starve while other workers have too many jobs enqueued,
• connector workers require another startup script, making deployment more difficult

It is fully compatible with the connector mechanism and only replaces workers.

How does it work?

• It starts as a thread in the Odoo main process
• It receives postgres NOTIFY messages each time jobs are added or updated in the queue_job table.
• It maintains an in-memory priority queue of jobs that is populated from the queue_job tables in all databases.
• It does not run jobs itself, but asks Odoo to run them through an anonymous /connector/runjob HTTP request. [1]

How to use it?

• Set the following environment variables:
  • ODOO_CONNECTOR_CHANNELS=root:4 (or any other channels configuration)
  • optional if xmlrpc_port is not set: ODOO_CONNECTOR_PORT=8069
• Or alternatively, set channels = root:4 in the [options-connector] section of the odoo configuration file.
• Start Odoo with --load=web,web_kanban,connector and --workers greater than 1. [2]
• Confirm the runner is starting correctly by checking the odoo log file:

...INFO...connector.jobrunner.runner: starting
...INFO...connector.jobrunner.runner: initializing database connections
...INFO...connector.jobrunner.runner: connector runner ready for db <dbname>
...INFO...connector.jobrunner.runner: database connections ready

• Disable the “Enqueue Jobs” cron.
• Do NOT start openerp-connector-worker.
• Create jobs (eg using base_import_async) and observe they start immediately and in parallel.
• Tip: to enable debug logging for the connector, use --log-handler=openerp.addons.connector:DEBUG

Caveat

• After creating a new database or installing connector on an existing database, Odoo must be restarted for the runner to detect it.
• When Odoo shuts down normally, it waits for running jobs to finish. However, when the Odoo server crashes or is otherwise force-stopped, running jobs are interrupted while the runner has no chance to know they have been aborted. In such situations, jobs may remain in started or enqueued state after the Odoo server is halted. Since the runner has no way to know if they are actually running or not, and does not know for sure if it is safe to restart the jobs, it does not attempt to restart them automatically. Such stale jobs therefore fill the running queue and prevent other jobs to start. You must therefore requeue them manually, either from the Jobs view, or by running the following SQL statement before starting Odoo:

update queue_job set state='pending' where state in ('started', 'enqueued')

Footnotes

[1]	From a security standpoint, it is safe to have an anonymous HTTP request because this request only accepts to run jobs that are enqueued.

[2]	It works with the threaded Odoo server too, although this way of running Odoo is obviously not for production purposes.

class connector.jobrunner.runner.ConnectorRunner(port=8069, channel_config_string='root:1')

Bases: object

close_databases(remove_jobs=True)

get_db_names()

initialize_databases()

process_notifications()

run()

run_jobs()

stop()

wait_notification()

class connector.jobrunner.runner.Database(db_name)

Bases: object

close()

select_jobs(where, args)

set_job_enqueued(uuid)