[ADD] bus_alt_connection
Nils Hamerlinck
Ioan Galan (Studio73)
parent
5377f90bc3
commit
16d4b194d7
|
|
@ -0,0 +1,187 @@
|
|||
==================
|
||||
Bus Alt Connection
|
||||
==================
|
||||
|
||||
.. !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
|
||||
!! This file is generated by oca-gen-addon-readme !!
|
||||
!! changes will be overwritten. !!
|
||||
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
|
||||
|
||||
.. |badge1| image:: https://img.shields.io/badge/maturity-Beta-yellow.png
|
||||
:target: https://odoo-community.org/page/development-status
|
||||
:alt: Beta
|
||||
.. |badge2| image:: https://img.shields.io/badge/licence-AGPL--3-blue.png
|
||||
:target: http://www.gnu.org/licenses/agpl-3.0-standalone.html
|
||||
:alt: License: AGPL-3
|
||||
.. |badge3| image:: https://img.shields.io/badge/github-OCA%2Fserver--tools-lightgray.png?logo=github
|
||||
:target: https://github.com/OCA/server-tools/tree/12.0/bus_alt_connection
|
||||
:alt: OCA/server-tools
|
||||
.. |badge4| image:: https://img.shields.io/badge/weblate-Translate%20me-F47D42.png
|
||||
:target: https://translation.odoo-community.org/projects/server-tools-12-0/server-tools-12-0-bus_alt_connection
|
||||
:alt: Translate me on Weblate
|
||||
.. |badge5| image:: https://img.shields.io/badge/runbot-Try%20me-875A7B.png
|
||||
:target: https://runbot.odoo-community.org/runbot/149/12.0
|
||||
:alt: Try me on Runbot
|
||||
|
||||
|badge1| |badge2| |badge3| |badge4| |badge5|
|
||||
|
||||
This module makes it possible to use PgBouncer_ as a connection pooler
|
||||
for odoo.
|
||||
|
||||
.. _PgBouncer: https://pgbouncer.github.io/
|
||||
|
||||
Why isn't odoo's connection pooling good enough?
|
||||
================================================
|
||||
|
||||
Odoo's builtin connection pooling works at process level: each Odoo process
|
||||
has its own ConnectionPool_, limited to ``db_maxconn``.
|
||||
|
||||
It does the job of re-using open connections available in the pool.
|
||||
But it never closes these connections, `unless reaching db_maxconn`_.
|
||||
|
||||
.. _ConnectionPool: https://github.com/odoo/odoo/blob/12.0/odoo/sql_db.py#L525
|
||||
.. _`unless reaching db_maxconn`: https://github.com/odoo/odoo/blob/12.0/odoo/sql_db.py#L593
|
||||
|
||||
In practice, we observe that each odoo worker will end up
|
||||
with up to 3 open connection in its pool.
|
||||
With 10 http workers, that's up to 30 connection continuously open just
|
||||
for one single instance.
|
||||
|
||||
Here comes PgBouncer
|
||||
====================
|
||||
|
||||
PgBouncer will help to limit this number of open connections,
|
||||
by sharing a pool of connections at the instance level, between
|
||||
all workers. Odoo workers will still have up to 3 open connections,
|
||||
but these will be connections to PgBouncer, that on its side will
|
||||
close unnecessary connections to pg.
|
||||
|
||||
This has proven to help performances on Odoo deployments with
|
||||
multiple instances.
|
||||
|
||||
It allows you to define how resources should be shared,
|
||||
according to your priorities, e.g. :
|
||||
|
||||
* key odoo instance on host A can open up to 30 connections
|
||||
* while odoo instance on host B, dedicated to reports,
|
||||
can open up to 10 connections only
|
||||
|
||||
And most importantly, it helps you to ensure that
|
||||
``max_connections`` will never be reached on pg server side.
|
||||
|
||||
|
||||
Why is this module needed?
|
||||
==========================
|
||||
|
||||
When configuring PgBouncer, you can choose between 2 transaction pooling modes:
|
||||
|
||||
* `pool_mode = session`
|
||||
* `pool_mode = transaction`
|
||||
|
||||
If we choose `pool_mode = session`, then one server connection will be tied
|
||||
to a given odoo process until its death, which is exactly what we're trying
|
||||
to change. Thus, to release the server connection once the transaction is
|
||||
complete, we use `pool_mode = transaction`.
|
||||
|
||||
This works fine, except for Odoo's longpolling features that relies
|
||||
on the `LISTEN/NOTIFY`_ mechanism from pg, which is `not compatible`_ with that
|
||||
mode.
|
||||
|
||||
.. _`LISTEN/NOTIFY`: https://www.postgresql.org/docs/9.6/static/sql-notify.html
|
||||
.. _`not compatible`: https://wiki.postgresql.org/wiki/PgBouncer
|
||||
|
||||
|
||||
To be more precise, `NOTIFY` statements are properly transfered by PgBouncer
|
||||
in that mode; only the `LISTEN` statement isn't (because it needs to keep the
|
||||
server connection open).
|
||||
|
||||
So for the unique "listening" connection per instance that requires this
|
||||
statement (here_), we need odoo to connect directly to the pg server, bypassing
|
||||
PgBouncer.
|
||||
|
||||
That's what this module implements, by overriding the relevant method
|
||||
of the Dispatcher_.
|
||||
|
||||
.. _here: https://github.com/odoo/odoo/blob/12.0/addons/bus/models/bus.py#L166
|
||||
.. _Dispatcher: https://github.com/odoo/odoo/blob/12.0/addons/bus/models/bus.py#L105
|
||||
|
||||
**Table of contents**
|
||||
|
||||
.. contents::
|
||||
:local:
|
||||
|
||||
Installation
|
||||
============
|
||||
|
||||
You don't need to install this module in the database(s) to enable it.
|
||||
|
||||
But you need to load it server-wide:
|
||||
|
||||
* By starting Odoo with ``--load=web,bus_alt_connection``
|
||||
|
||||
* Or by updating its configuration file:
|
||||
|
||||
.. code-block:: ini
|
||||
|
||||
[options]
|
||||
(...)
|
||||
server_wide_modules = web,bus_alt_connection
|
||||
|
||||
Configuration
|
||||
=============
|
||||
|
||||
You need to define how to connect directly to the database:
|
||||
|
||||
* Either by defining environment variables:
|
||||
|
||||
- ``IMDISPATCHER_DB_HOST=db-01``
|
||||
- ``IMDISPATCHER_DB_PORT=5432``
|
||||
|
||||
* Or in Odoo's configuration file:
|
||||
|
||||
.. code-block:: ini
|
||||
|
||||
[options]
|
||||
(...)
|
||||
imdispatcher_db_host = db-01
|
||||
imdispatcher_db_port = 5432
|
||||
|
||||
Bug Tracker
|
||||
===========
|
||||
|
||||
Bugs are tracked on `GitHub Issues <https://github.com/OCA/server-tools/issues>`_.
|
||||
In case of trouble, please check there if your issue has already been reported.
|
||||
If you spotted it first, help us smashing it by providing a detailed and welcomed
|
||||
`feedback <https://github.com/OCA/server-tools/issues/new?body=module:%20bus_alt_connection%0Aversion:%2012.0%0A%0A**Steps%20to%20reproduce**%0A-%20...%0A%0A**Current%20behavior**%0A%0A**Expected%20behavior**>`_.
|
||||
|
||||
Do not contact contributors directly about support or help with technical issues.
|
||||
|
||||
Credits
|
||||
=======
|
||||
|
||||
Authors
|
||||
~~~~~~~
|
||||
|
||||
* Trobz
|
||||
|
||||
Contributors
|
||||
~~~~~~~~~~~~
|
||||
|
||||
* Nils Hamerlinck <nils@trobz.com>
|
||||
|
||||
Maintainers
|
||||
~~~~~~~~~~~
|
||||
|
||||
This module is maintained by the OCA.
|
||||
|
||||
.. image:: https://odoo-community.org/logo.png
|
||||
:alt: Odoo Community Association
|
||||
:target: https://odoo-community.org
|
||||
|
||||
OCA, or the Odoo Community Association, is a nonprofit organization whose
|
||||
mission is to support the collaborative development of Odoo features and
|
||||
promote its widespread use.
|
||||
|
||||
This module is part of the `OCA/server-tools <https://github.com/OCA/server-tools/tree/12.0/bus_alt_connection>`_ project on GitHub.
|
||||
|
||||
You are welcome to contribute. To learn how please visit https://odoo-community.org/page/Contribute.
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
# Copyright 2019 Trobz <https://trobz.com>
|
||||
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl).
|
||||
from . import models
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
# Copyright 2019 Trobz <https://trobz.com>
|
||||
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl).
|
||||
|
||||
{
|
||||
'name': 'Bus Alt Connection',
|
||||
'summary': 'Needed when using PgBouncer as a connection pooler',
|
||||
'version': '12.0.1.0.0',
|
||||
'author': 'Trobz,Odoo Community Association (OCA)',
|
||||
'website': 'https://github.com/OCA/server-tools',
|
||||
'category': 'Extra Tools',
|
||||
'license': 'AGPL-3',
|
||||
'depends': ['bus'],
|
||||
'installable': True,
|
||||
'auto_install': False,
|
||||
}
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
# Copyright 2019 Trobz <https://trobz.com>
|
||||
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl).
|
||||
from . import bus
|
||||
|
|
@ -0,0 +1,77 @@
|
|||
# Copyright 2019 Trobz <https://trobz.com>
|
||||
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl).
|
||||
|
||||
import os
|
||||
import json
|
||||
import logging
|
||||
import select
|
||||
|
||||
import psycopg2
|
||||
from psycopg2.extensions import ISOLATION_LEVEL_AUTOCOMMIT
|
||||
|
||||
import odoo
|
||||
from odoo.tools import config
|
||||
|
||||
from odoo.addons.bus.models.bus import hashable, TIMEOUT
|
||||
import odoo.addons.bus.models.bus
|
||||
|
||||
|
||||
_logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _connection_info_for(db_name):
|
||||
db_or_uri, connection_info = odoo.sql_db.connection_info_for(db_name)
|
||||
|
||||
for p in ('host', 'port'):
|
||||
cfg = (os.environ.get('ODOO_IMDISPATCHER_DB_%s' % p.upper()) or
|
||||
config.get('imdispatcher_db_' + p))
|
||||
|
||||
if cfg:
|
||||
connection_info[p] = cfg
|
||||
|
||||
return connection_info
|
||||
|
||||
|
||||
class ImDispatch(odoo.addons.bus.models.bus.ImDispatch):
|
||||
|
||||
def loop(self):
|
||||
""" Dispatch postgres notifications to the relevant
|
||||
polling threads/greenlets """
|
||||
connection_info = _connection_info_for('postgres')
|
||||
_logger.info("Bus.loop listen imbus on db postgres "
|
||||
"(via %(host)s:%(port)s)",
|
||||
connection_info)
|
||||
conn = psycopg2.connect(**connection_info)
|
||||
conn.set_isolation_level(ISOLATION_LEVEL_AUTOCOMMIT)
|
||||
with conn.cursor() as cr:
|
||||
cr.execute("listen imbus")
|
||||
conn.commit()
|
||||
while True:
|
||||
if select.select([conn], [], [], TIMEOUT) == ([], [], []):
|
||||
pass
|
||||
else:
|
||||
conn.poll()
|
||||
channels = []
|
||||
while conn.notifies:
|
||||
channels.extend(json.loads(conn
|
||||
.notifies.pop().payload))
|
||||
# dispatch to local threads/greenlets
|
||||
events = set()
|
||||
for channel in channels:
|
||||
events.update(self.channels.pop(hashable(channel),
|
||||
set()))
|
||||
for event in events:
|
||||
event.set()
|
||||
|
||||
|
||||
odoo.addons.bus.models.bus.ImDispatch = ImDispatch
|
||||
|
||||
# we can replace the existing dispatcher because its thread
|
||||
# has not been started yet; indeed, since a2ed3d it only starts
|
||||
# on first /poll request:
|
||||
# https://github.com/odoo/odoo/commit/a2ed3d3d5bdb6025a1ba14ad557a115a86413e65
|
||||
|
||||
if not odoo.multi_process or odoo.evented:
|
||||
dispatch = ImDispatch()
|
||||
odoo.addons.bus.models.bus.dispatch = dispatch
|
||||
odoo.addons.bus.controllers.main.dispatch = dispatch
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
You need to define how to connect directly to the database:
|
||||
|
||||
* Either by defining environment variables:
|
||||
|
||||
- ``IMDISPATCHER_DB_HOST=db-01``
|
||||
- ``IMDISPATCHER_DB_PORT=5432``
|
||||
|
||||
* Or in Odoo's configuration file:
|
||||
|
||||
.. code-block:: ini
|
||||
|
||||
[options]
|
||||
(...)
|
||||
imdispatcher_db_host = db-01
|
||||
imdispatcher_db_port = 5432
|
||||
|
|
@ -0,0 +1 @@
|
|||
* Nils Hamerlinck <nils@trobz.com>
|
||||
|
|
@ -0,0 +1,79 @@
|
|||
This module makes it possible to use PgBouncer_ as a connection pooler
|
||||
for odoo.
|
||||
|
||||
.. _PgBouncer: https://pgbouncer.github.io/
|
||||
|
||||
Why isn't odoo's connection pooling good enough?
|
||||
================================================
|
||||
|
||||
Odoo's builtin connection pooling works at process level: each Odoo process
|
||||
has its own ConnectionPool_, limited to ``db_maxconn``.
|
||||
|
||||
It does the job of re-using open connections available in the pool.
|
||||
But it never closes these connections, `unless reaching db_maxconn`_.
|
||||
|
||||
.. _ConnectionPool: https://github.com/odoo/odoo/blob/12.0/odoo/sql_db.py#L525
|
||||
.. _`unless reaching db_maxconn`: https://github.com/odoo/odoo/blob/12.0/odoo/sql_db.py#L593
|
||||
|
||||
In practice, we observe that each odoo worker will end up
|
||||
with up to 3 open connection in its pool.
|
||||
With 10 http workers, that's up to 30 connection continuously open just
|
||||
for one single instance.
|
||||
|
||||
Here comes PgBouncer
|
||||
====================
|
||||
|
||||
PgBouncer will help to limit this number of open connections,
|
||||
by sharing a pool of connections at the instance level, between
|
||||
all workers. Odoo workers will still have up to 3 open connections,
|
||||
but these will be connections to PgBouncer, that on its side will
|
||||
close unnecessary connections to pg.
|
||||
|
||||
This has proven to help performances on Odoo deployments with
|
||||
multiple instances.
|
||||
|
||||
It allows you to define how resources should be shared,
|
||||
according to your priorities, e.g. :
|
||||
|
||||
* key odoo instance on host A can open up to 30 connections
|
||||
* while odoo instance on host B, dedicated to reports,
|
||||
can open up to 10 connections only
|
||||
|
||||
And most importantly, it helps you to ensure that
|
||||
``max_connections`` will never be reached on pg server side.
|
||||
|
||||
|
||||
Why is this module needed?
|
||||
==========================
|
||||
|
||||
When configuring PgBouncer, you can choose between 2 transaction pooling modes:
|
||||
|
||||
* `pool_mode = session`
|
||||
* `pool_mode = transaction`
|
||||
|
||||
If we choose `pool_mode = session`, then one server connection will be tied
|
||||
to a given odoo process until its death, which is exactly what we're trying
|
||||
to change. Thus, to release the server connection once the transaction is
|
||||
complete, we use `pool_mode = transaction`.
|
||||
|
||||
This works fine, except for Odoo's longpolling features that relies
|
||||
on the `LISTEN/NOTIFY`_ mechanism from pg, which is `not compatible`_ with that
|
||||
mode.
|
||||
|
||||
.. _`LISTEN/NOTIFY`: https://www.postgresql.org/docs/9.6/static/sql-notify.html
|
||||
.. _`not compatible`: https://wiki.postgresql.org/wiki/PgBouncer
|
||||
|
||||
|
||||
To be more precise, `NOTIFY` statements are properly transfered by PgBouncer
|
||||
in that mode; only the `LISTEN` statement isn't (because it needs to keep the
|
||||
server connection open).
|
||||
|
||||
So for the unique "listening" connection per instance that requires this
|
||||
statement (here_), we need odoo to connect directly to the pg server, bypassing
|
||||
PgBouncer.
|
||||
|
||||
That's what this module implements, by overriding the relevant method
|
||||
of the Dispatcher_.
|
||||
|
||||
.. _here: https://github.com/odoo/odoo/blob/12.0/addons/bus/models/bus.py#L166
|
||||
.. _Dispatcher: https://github.com/odoo/odoo/blob/12.0/addons/bus/models/bus.py#L105
|
||||
|
|
@ -0,0 +1,13 @@
|
|||
You don't need to install this module in the database(s) to enable it.
|
||||
|
||||
But you need to load it server-wide:
|
||||
|
||||
* By starting Odoo with ``--load=web,bus_alt_connection``
|
||||
|
||||
* Or by updating its configuration file:
|
||||
|
||||
.. code-block:: ini
|
||||
|
||||
[options]
|
||||
(...)
|
||||
server_wide_modules = web,bus_alt_connection
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 9.2 KiB |
|
|
@ -0,0 +1,498 @@
|
|||
<?xml version="1.0" encoding="utf-8" ?>
|
||||
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
|
||||
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
|
||||
<head>
|
||||
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
|
||||
<meta name="generator" content="Docutils 0.15.1: http://docutils.sourceforge.net/" />
|
||||
<title>Bus Alt Connection</title>
|
||||
<style type="text/css">
|
||||
|
||||
/*
|
||||
:Author: David Goodger (goodger@python.org)
|
||||
:Id: $Id: html4css1.css 7952 2016-07-26 18:15:59Z milde $
|
||||
:Copyright: This stylesheet has been placed in the public domain.
|
||||
|
||||
Default cascading style sheet for the HTML output of Docutils.
|
||||
|
||||
See http://docutils.sf.net/docs/howto/html-stylesheets.html for how to
|
||||
customize this style sheet.
|
||||
*/
|
||||
|
||||
/* used to remove borders from tables and images */
|
||||
.borderless, table.borderless td, table.borderless th {
|
||||
border: 0 }
|
||||
|
||||
table.borderless td, table.borderless th {
|
||||
/* Override padding for "table.docutils td" with "! important".
|
||||
The right padding separates the table cells. */
|
||||
padding: 0 0.5em 0 0 ! important }
|
||||
|
||||
.first {
|
||||
/* Override more specific margin styles with "! important". */
|
||||
margin-top: 0 ! important }
|
||||
|
||||
.last, .with-subtitle {
|
||||
margin-bottom: 0 ! important }
|
||||
|
||||
.hidden {
|
||||
display: none }
|
||||
|
||||
.subscript {
|
||||
vertical-align: sub;
|
||||
font-size: smaller }
|
||||
|
||||
.superscript {
|
||||
vertical-align: super;
|
||||
font-size: smaller }
|
||||
|
||||
a.toc-backref {
|
||||
text-decoration: none ;
|
||||
color: black }
|
||||
|
||||
blockquote.epigraph {
|
||||
margin: 2em 5em ; }
|
||||
|
||||
dl.docutils dd {
|
||||
margin-bottom: 0.5em }
|
||||
|
||||
object[type="image/svg+xml"], object[type="application/x-shockwave-flash"] {
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
/* Uncomment (and remove this text!) to get bold-faced definition list terms
|
||||
dl.docutils dt {
|
||||
font-weight: bold }
|
||||
*/
|
||||
|
||||
div.abstract {
|
||||
margin: 2em 5em }
|
||||
|
||||
div.abstract p.topic-title {
|
||||
font-weight: bold ;
|
||||
text-align: center }
|
||||
|
||||
div.admonition, div.attention, div.caution, div.danger, div.error,
|
||||
div.hint, div.important, div.note, div.tip, div.warning {
|
||||
margin: 2em ;
|
||||
border: medium outset ;
|
||||
padding: 1em }
|
||||
|
||||
div.admonition p.admonition-title, div.hint p.admonition-title,
|
||||
div.important p.admonition-title, div.note p.admonition-title,
|
||||
div.tip p.admonition-title {
|
||||
font-weight: bold ;
|
||||
font-family: sans-serif }
|
||||
|
||||
div.attention p.admonition-title, div.caution p.admonition-title,
|
||||
div.danger p.admonition-title, div.error p.admonition-title,
|
||||
div.warning p.admonition-title, .code .error {
|
||||
color: red ;
|
||||
font-weight: bold ;
|
||||
font-family: sans-serif }
|
||||
|
||||
/* Uncomment (and remove this text!) to get reduced vertical space in
|
||||
compound paragraphs.
|
||||
div.compound .compound-first, div.compound .compound-middle {
|
||||
margin-bottom: 0.5em }
|
||||
|
||||
div.compound .compound-last, div.compound .compound-middle {
|
||||
margin-top: 0.5em }
|
||||
*/
|
||||
|
||||
div.dedication {
|
||||
margin: 2em 5em ;
|
||||
text-align: center ;
|
||||
font-style: italic }
|
||||
|
||||
div.dedication p.topic-title {
|
||||
font-weight: bold ;
|
||||
font-style: normal }
|
||||
|
||||
div.figure {
|
||||
margin-left: 2em ;
|
||||
margin-right: 2em }
|
||||
|
||||
div.footer, div.header {
|
||||
clear: both;
|
||||
font-size: smaller }
|
||||
|
||||
div.line-block {
|
||||
display: block ;
|
||||
margin-top: 1em ;
|
||||
margin-bottom: 1em }
|
||||
|
||||
div.line-block div.line-block {
|
||||
margin-top: 0 ;
|
||||
margin-bottom: 0 ;
|
||||
margin-left: 1.5em }
|
||||
|
||||
div.sidebar {
|
||||
margin: 0 0 0.5em 1em ;
|
||||
border: medium outset ;
|
||||
padding: 1em ;
|
||||
background-color: #ffffee ;
|
||||
width: 40% ;
|
||||
float: right ;
|
||||
clear: right }
|
||||
|
||||
div.sidebar p.rubric {
|
||||
font-family: sans-serif ;
|
||||
font-size: medium }
|
||||
|
||||
div.system-messages {
|
||||
margin: 5em }
|
||||
|
||||
div.system-messages h1 {
|
||||
color: red }
|
||||
|
||||
div.system-message {
|
||||
border: medium outset ;
|
||||
padding: 1em }
|
||||
|
||||
div.system-message p.system-message-title {
|
||||
color: red ;
|
||||
font-weight: bold }
|
||||
|
||||
div.topic {
|
||||
margin: 2em }
|
||||
|
||||
h1.section-subtitle, h2.section-subtitle, h3.section-subtitle,
|
||||
h4.section-subtitle, h5.section-subtitle, h6.section-subtitle {
|
||||
margin-top: 0.4em }
|
||||
|
||||
h1.title {
|
||||
text-align: center }
|
||||
|
||||
h2.subtitle {
|
||||
text-align: center }
|
||||
|
||||
hr.docutils {
|
||||
width: 75% }
|
||||
|
||||
img.align-left, .figure.align-left, object.align-left, table.align-left {
|
||||
clear: left ;
|
||||
float: left ;
|
||||
margin-right: 1em }
|
||||
|
||||
img.align-right, .figure.align-right, object.align-right, table.align-right {
|
||||
clear: right ;
|
||||
float: right ;
|
||||
margin-left: 1em }
|
||||
|
||||
img.align-center, .figure.align-center, object.align-center {
|
||||
display: block;
|
||||
margin-left: auto;
|
||||
margin-right: auto;
|
||||
}
|
||||
|
||||
table.align-center {
|
||||
margin-left: auto;
|
||||
margin-right: auto;
|
||||
}
|
||||
|
||||
.align-left {
|
||||
text-align: left }
|
||||
|
||||
.align-center {
|
||||
clear: both ;
|
||||
text-align: center }
|
||||
|
||||
.align-right {
|
||||
text-align: right }
|
||||
|
||||
/* reset inner alignment in figures */
|
||||
div.align-right {
|
||||
text-align: inherit }
|
||||
|
||||
/* div.align-center * { */
|
||||
/* text-align: left } */
|
||||
|
||||
.align-top {
|
||||
vertical-align: top }
|
||||
|
||||
.align-middle {
|
||||
vertical-align: middle }
|
||||
|
||||
.align-bottom {
|
||||
vertical-align: bottom }
|
||||
|
||||
ol.simple, ul.simple {
|
||||
margin-bottom: 1em }
|
||||
|
||||
ol.arabic {
|
||||
list-style: decimal }
|
||||
|
||||
ol.loweralpha {
|
||||
list-style: lower-alpha }
|
||||
|
||||
ol.upperalpha {
|
||||
list-style: upper-alpha }
|
||||
|
||||
ol.lowerroman {
|
||||
list-style: lower-roman }
|
||||
|
||||
ol.upperroman {
|
||||
list-style: upper-roman }
|
||||
|
||||
p.attribution {
|
||||
text-align: right ;
|
||||
margin-left: 50% }
|
||||
|
||||
p.caption {
|
||||
font-style: italic }
|
||||
|
||||
p.credits {
|
||||
font-style: italic ;
|
||||
font-size: smaller }
|
||||
|
||||
p.label {
|
||||
white-space: nowrap }
|
||||
|
||||
p.rubric {
|
||||
font-weight: bold ;
|
||||
font-size: larger ;
|
||||
color: maroon ;
|
||||
text-align: center }
|
||||
|
||||
p.sidebar-title {
|
||||
font-family: sans-serif ;
|
||||
font-weight: bold ;
|
||||
font-size: larger }
|
||||
|
||||
p.sidebar-subtitle {
|
||||
font-family: sans-serif ;
|
||||
font-weight: bold }
|
||||
|
||||
p.topic-title {
|
||||
font-weight: bold }
|
||||
|
||||
pre.address {
|
||||
margin-bottom: 0 ;
|
||||
margin-top: 0 ;
|
||||
font: inherit }
|
||||
|
||||
pre.literal-block, pre.doctest-block, pre.math, pre.code {
|
||||
margin-left: 2em ;
|
||||
margin-right: 2em }
|
||||
|
||||
pre.code .ln { color: grey; } /* line numbers */
|
||||
pre.code, code { background-color: #eeeeee }
|
||||
pre.code .comment, code .comment { color: #5C6576 }
|
||||
pre.code .keyword, code .keyword { color: #3B0D06; font-weight: bold }
|
||||
pre.code .literal.string, code .literal.string { color: #0C5404 }
|
||||
pre.code .name.builtin, code .name.builtin { color: #352B84 }
|
||||
pre.code .deleted, code .deleted { background-color: #DEB0A1}
|
||||
pre.code .inserted, code .inserted { background-color: #A3D289}
|
||||
|
||||
span.classifier {
|
||||
font-family: sans-serif ;
|
||||
font-style: oblique }
|
||||
|
||||
span.classifier-delimiter {
|
||||
font-family: sans-serif ;
|
||||
font-weight: bold }
|
||||
|
||||
span.interpreted {
|
||||
font-family: sans-serif }
|
||||
|
||||
span.option {
|
||||
white-space: nowrap }
|
||||
|
||||
span.pre {
|
||||
white-space: pre }
|
||||
|
||||
span.problematic {
|
||||
color: red }
|
||||
|
||||
span.section-subtitle {
|
||||
/* font-size relative to parent (h1..h6 element) */
|
||||
font-size: 80% }
|
||||
|
||||
table.citation {
|
||||
border-left: solid 1px gray;
|
||||
margin-left: 1px }
|
||||
|
||||
table.docinfo {
|
||||
margin: 2em 4em }
|
||||
|
||||
table.docutils {
|
||||
margin-top: 0.5em ;
|
||||
margin-bottom: 0.5em }
|
||||
|
||||
table.footnote {
|
||||
border-left: solid 1px black;
|
||||
margin-left: 1px }
|
||||
|
||||
table.docutils td, table.docutils th,
|
||||
table.docinfo td, table.docinfo th {
|
||||
padding-left: 0.5em ;
|
||||
padding-right: 0.5em ;
|
||||
vertical-align: top }
|
||||
|
||||
table.docutils th.field-name, table.docinfo th.docinfo-name {
|
||||
font-weight: bold ;
|
||||
text-align: left ;
|
||||
white-space: nowrap ;
|
||||
padding-left: 0 }
|
||||
|
||||
/* "booktabs" style (no vertical lines) */
|
||||
table.docutils.booktabs {
|
||||
border: 0px;
|
||||
border-top: 2px solid;
|
||||
border-bottom: 2px solid;
|
||||
border-collapse: collapse;
|
||||
}
|
||||
table.docutils.booktabs * {
|
||||
border: 0px;
|
||||
}
|
||||
table.docutils.booktabs th {
|
||||
border-bottom: thin solid;
|
||||
text-align: left;
|
||||
}
|
||||
|
||||
h1 tt.docutils, h2 tt.docutils, h3 tt.docutils,
|
||||
h4 tt.docutils, h5 tt.docutils, h6 tt.docutils {
|
||||
font-size: 100% }
|
||||
|
||||
ul.auto-toc {
|
||||
list-style-type: none }
|
||||
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="document" id="bus-alt-connection">
|
||||
<h1 class="title">Bus Alt Connection</h1>
|
||||
|
||||
<!-- !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
|
||||
!! This file is generated by oca-gen-addon-readme !!
|
||||
!! changes will be overwritten. !!
|
||||
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! -->
|
||||
<p><a class="reference external" href="https://odoo-community.org/page/development-status"><img alt="Beta" src="https://img.shields.io/badge/maturity-Beta-yellow.png" /></a> <a class="reference external" href="http://www.gnu.org/licenses/agpl-3.0-standalone.html"><img alt="License: AGPL-3" src="https://img.shields.io/badge/licence-AGPL--3-blue.png" /></a> <a class="reference external" href="https://github.com/OCA/server-tools/tree/12.0/bus_alt_connection"><img alt="OCA/server-tools" src="https://img.shields.io/badge/github-OCA%2Fserver--tools-lightgray.png?logo=github" /></a> <a class="reference external" href="https://translation.odoo-community.org/projects/server-tools-12-0/server-tools-12-0-bus_alt_connection"><img alt="Translate me on Weblate" src="https://img.shields.io/badge/weblate-Translate%20me-F47D42.png" /></a> <a class="reference external" href="https://runbot.odoo-community.org/runbot/149/12.0"><img alt="Try me on Runbot" src="https://img.shields.io/badge/runbot-Try%20me-875A7B.png" /></a></p>
|
||||
<p>This module makes it possible to use <a class="reference external" href="https://pgbouncer.github.io/">PgBouncer</a> as a connection pooler
|
||||
for odoo.</p>
|
||||
<div class="section" id="why-isn-t-odoo-s-connection-pooling-good-enough">
|
||||
<h1>Why isn’t odoo’s connection pooling good enough?</h1>
|
||||
<p>Odoo’s builtin connection pooling works at process level: each Odoo process
|
||||
has its own <a class="reference external" href="https://github.com/odoo/odoo/blob/12.0/odoo/sql_db.py#L525">ConnectionPool</a>, limited to <tt class="docutils literal">db_maxconn</tt>.</p>
|
||||
<p>It does the job of re-using open connections available in the pool.
|
||||
But it never closes these connections, <a class="reference external" href="https://github.com/odoo/odoo/blob/12.0/odoo/sql_db.py#L593">unless reaching db_maxconn</a>.</p>
|
||||
<p>In practice, we observe that each odoo worker will end up
|
||||
with up to 3 open connection in its pool.
|
||||
With 10 http workers, that’s up to 30 connection continuously open just
|
||||
for one single instance.</p>
|
||||
</div>
|
||||
<div class="section" id="here-comes-pgbouncer">
|
||||
<h1>Here comes PgBouncer</h1>
|
||||
<p>PgBouncer will help to limit this number of open connections,
|
||||
by sharing a pool of connections at the instance level, between
|
||||
all workers. Odoo workers will still have up to 3 open connections,
|
||||
but these will be connections to PgBouncer, that on its side will
|
||||
close unnecessary connections to pg.</p>
|
||||
<p>This has proven to help performances on Odoo deployments with
|
||||
multiple instances.</p>
|
||||
<p>It allows you to define how resources should be shared,
|
||||
according to your priorities, e.g. :</p>
|
||||
<ul class="simple">
|
||||
<li>key odoo instance on host A can open up to 30 connections</li>
|
||||
<li>while odoo instance on host B, dedicated to reports,
|
||||
can open up to 10 connections only</li>
|
||||
</ul>
|
||||
<p>And most importantly, it helps you to ensure that
|
||||
<tt class="docutils literal">max_connections</tt> will never be reached on pg server side.</p>
|
||||
</div>
|
||||
<div class="section" id="why-is-this-module-needed">
|
||||
<h1>Why is this module needed?</h1>
|
||||
<p>When configuring PgBouncer, you can choose between 2 transaction pooling modes:</p>
|
||||
<ul class="simple">
|
||||
<li><cite>pool_mode = session</cite></li>
|
||||
<li><cite>pool_mode = transaction</cite></li>
|
||||
</ul>
|
||||
<p>If we choose <cite>pool_mode = session</cite>, then one server connection will be tied
|
||||
to a given odoo process until its death, which is exactly what we’re trying
|
||||
to change. Thus, to release the server connection once the transaction is
|
||||
complete, we use <cite>pool_mode = transaction</cite>.</p>
|
||||
<p>This works fine, except for Odoo’s longpolling features that relies
|
||||
on the <a class="reference external" href="https://www.postgresql.org/docs/9.6/static/sql-notify.html">LISTEN/NOTIFY</a> mechanism from pg, which is <a class="reference external" href="https://wiki.postgresql.org/wiki/PgBouncer">not compatible</a> with that
|
||||
mode.</p>
|
||||
<p>To be more precise, <cite>NOTIFY</cite> statements are properly transfered by PgBouncer
|
||||
in that mode; only the <cite>LISTEN</cite> statement isn’t (because it needs to keep the
|
||||
server connection open).</p>
|
||||
<p>So for the unique “listening” connection per instance that requires this
|
||||
statement (<a class="reference external" href="https://github.com/odoo/odoo/blob/12.0/addons/bus/models/bus.py#L166">here</a>), we need odoo to connect directly to the pg server, bypassing
|
||||
PgBouncer.</p>
|
||||
<p>That’s what this module implements, by overriding the relevant method
|
||||
of the <a class="reference external" href="https://github.com/odoo/odoo/blob/12.0/addons/bus/models/bus.py#L105">Dispatcher</a>.</p>
|
||||
<p><strong>Table of contents</strong></p>
|
||||
</div>
|
||||
<div class="section" id="installation">
|
||||
<h1>Installation</h1>
|
||||
<p>You don’t need to install this module in the database(s) to enable it.</p>
|
||||
<p>But you need to load it server-wide:</p>
|
||||
<ul class="simple">
|
||||
<li>By starting Odoo with <tt class="docutils literal"><span class="pre">--load=web,bus_alt_connection</span></tt></li>
|
||||
<li>Or by updating its configuration file:</li>
|
||||
</ul>
|
||||
<pre class="code ini literal-block">
|
||||
<span class="k">[options]</span>
|
||||
<span class="na">(...)</span>
|
||||
<span class="na">server_wide_modules</span> <span class="o">=</span> <span class="s">web,bus_alt_connection</span>
|
||||
</pre>
|
||||
</div>
|
||||
<div class="section" id="configuration">
|
||||
<h1>Configuration</h1>
|
||||
<p>You need to define how to connect directly to the database:</p>
|
||||
<ul>
|
||||
<li><p class="first">Either by defining environment variables:</p>
|
||||
<blockquote>
|
||||
<ul class="simple">
|
||||
<li><tt class="docutils literal"><span class="pre">IMDISPATCHER_DB_HOST=db-01</span></tt></li>
|
||||
<li><tt class="docutils literal">IMDISPATCHER_DB_PORT=5432</tt></li>
|
||||
</ul>
|
||||
</blockquote>
|
||||
</li>
|
||||
<li><p class="first">Or in Odoo’s configuration file:</p>
|
||||
</li>
|
||||
</ul>
|
||||
<pre class="code ini literal-block">
|
||||
<span class="k">[options]</span>
|
||||
<span class="na">(...)</span>
|
||||
<span class="na">imdispatcher_db_host</span> <span class="o">=</span> <span class="s">db-01</span>
|
||||
<span class="na">imdispatcher_db_port</span> <span class="o">=</span> <span class="s">5432</span>
|
||||
</pre>
|
||||
</div>
|
||||
<div class="section" id="bug-tracker">
|
||||
<h1>Bug Tracker</h1>
|
||||
<p>Bugs are tracked on <a class="reference external" href="https://github.com/OCA/server-tools/issues">GitHub Issues</a>.
|
||||
In case of trouble, please check there if your issue has already been reported.
|
||||
If you spotted it first, help us smashing it by providing a detailed and welcomed
|
||||
<a class="reference external" href="https://github.com/OCA/server-tools/issues/new?body=module:%20bus_alt_connection%0Aversion:%2012.0%0A%0A**Steps%20to%20reproduce**%0A-%20...%0A%0A**Current%20behavior**%0A%0A**Expected%20behavior**">feedback</a>.</p>
|
||||
<p>Do not contact contributors directly about support or help with technical issues.</p>
|
||||
</div>
|
||||
<div class="section" id="credits">
|
||||
<h1>Credits</h1>
|
||||
<div class="section" id="authors">
|
||||
<h2>Authors</h2>
|
||||
<ul class="simple">
|
||||
<li>Trobz</li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="section" id="contributors">
|
||||
<h2>Contributors</h2>
|
||||
<ul class="simple">
|
||||
<li>Nils Hamerlinck <<a class="reference external" href="mailto:nils@trobz.com">nils@trobz.com</a>></li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="section" id="maintainers">
|
||||
<h2>Maintainers</h2>
|
||||
<p>This module is maintained by the OCA.</p>
|
||||
<a class="reference external image-reference" href="https://odoo-community.org"><img alt="Odoo Community Association" src="https://odoo-community.org/logo.png" /></a>
|
||||
<p>OCA, or the Odoo Community Association, is a nonprofit organization whose
|
||||
mission is to support the collaborative development of Odoo features and
|
||||
promote its widespread use.</p>
|
||||
<p>This module is part of the <a class="reference external" href="https://github.com/OCA/server-tools/tree/12.0/bus_alt_connection">OCA/server-tools</a> project on GitHub.</p>
|
||||
<p>You are welcome to contribute. To learn how please visit <a class="reference external" href="https://odoo-community.org/page/Contribute">https://odoo-community.org/page/Contribute</a>.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
Loading…
Reference in New Issue