Techsystech
/
server-tools
mirror of https://github.com/OCA/server-tools.git
3
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

[MIG] base_import_odoo: Migration to v13

pull/3136/head
Holger Brunn 2020-12-07 07:40:23 +01:00 committed by Tom
parent 2a111a42c8
commit 70d4fecb2f
14 changed files with 660 additions and 100 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
base_import_odoo/CONTRIBUTORS.rst
View File

@ -1 +0,0 @@
* Holger Brunn <hbrunn@therp.nl>

121
base_import_odoo/README.rst
View File

@ -1 +1,120 @@
/
================
Import from Odoo
================
.. !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!! This file is generated by oca-gen-addon-readme !!
!! changes will be overwritten. !!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
.. |badge1| image:: https://img.shields.io/badge/maturity-Beta-yellow.png
:target: https://odoo-community.org/page/development-status
:alt: Beta
.. |badge2| image:: https://img.shields.io/badge/licence-AGPL--3-blue.png
:target: http://www.gnu.org/licenses/agpl-3.0-standalone.html
:alt: License: AGPL-3
.. |badge3| image:: https://img.shields.io/badge/github-OCA%2Fserver--tools-lightgray.png?logo=github
:target: https://github.com/OCA/server-tools/tree/13.0/base_import_odoo
:alt: OCA/server-tools
.. |badge4| image:: https://img.shields.io/badge/weblate-Translate%20me-F47D42.png
:target: https://translation.odoo-community.org/projects/server-tools-13-0/server-tools-13-0-base_import_odoo
:alt: Translate me on Weblate
.. |badge5| image:: https://img.shields.io/badge/runbot-Try%20me-875A7B.png
:target: https://runbot.odoo-community.org/runbot/149/13.0
:alt: Try me on Runbot
|badge1| |badge2| |badge3| |badge4| |badge5|
This module was written to import data from another Odoo database. The idea is that you define which models to import from the other database, and add eventual mappings for records you don't want to import.
Use cases
=========
- merging databases
- one way sync
- aggregating management data from distributed systems
**Table of contents**
.. contents::
:local:
Configuration
=============
Go to Settings / Remote Odoo import / Import configurations and create a configuration.
After filling in your credentials, select models you want to import from the remote database. If you only want to import a subset of the records, add an appropriate domain.
The import will copy records of all models listed, and handle links to records of models which are not imported depending on the existing field mappings. Field mappings to local records also are a stopping condition. Without those, the import will have to create some record for all required x2x fields, which you probably don't want.
Probably you'll want to map records of model ``res.company``, and at least the admin user.
The module doesn't import one2many fields, if you want to have those, add the model the field in question points to to the list of imported models, possibly with a domain.
If you don't fill in a remote ID, the addon will use the configured local ID for every record of the model (this way, you can for example map all users in the remote system to some import user in the current system).
For fields that have a uniqueness constraint (like ``res.users#login``), create a field mapping if type ``unique``, then the import will generate a unique value for this field.
For models using references with two fields (like ``ir.attachment``), create a field mapping of type ``by reference`` and select the two fields involved. The import will transform known ids (=ids of models you import) to the respective local id, and clean out the model/id fields for unknown models/ids.
You can add custom defaults per model in case your database has different required fields than the source database. For ``res.partner``, you'll most certainly fill in ``{'name': '/'}`` or somethign similar.
Usage
=====
To use this module, you need to:
#. go to an import configuration and hit the button ``Run import``
#. be patient, this creates a cronjob which will start up to a minutes afterwards
#. reload the form, as soon as the cronjob runs you'll see a field ``Progress`` that lets you inspect what was imported already
#. note that the cronjob also resets the password as soon as it has read it. So for a subsequent import, you'll have to fill it in again
#. running an import a second time won't duplicate data, it should recognize records imported earlier and just update them
Known issues / Roadmap
======================
* Yes of course this duplicates a lot of connector functionality. Rewrite this with the connector framework, probably collaborate with https://github.com/OCA/connector-odoo2odoo
* Support reference fields, while being at it refactor _run_import_map_values to call a function per field type
* Add duplicate handling strategy 'Overwrite older'
Bug Tracker
===========
Bugs are tracked on `GitHub Issues <https://github.com/OCA/server-tools/issues>`_.
In case of trouble, please check there if your issue has already been reported.
If you spotted it first, help us smashing it by providing a detailed and welcomed
`feedback <https://github.com/OCA/server-tools/issues/new?body=module:%20base_import_odoo%0Aversion:%2013.0%0A%0A**Steps%20to%20reproduce**%0A-%20...%0A%0A**Current%20behavior**%0A%0A**Expected%20behavior**>`_.
Do not contact contributors directly about support or help with technical issues.
Credits
=======
Authors
~~~~~~~
* Therp BV
* Hunki Enterprises BV
Contributors
~~~~~~~~~~~~
* Holger Brunn <mail@hunki-enterprises.com>
Maintainers
~~~~~~~~~~~
This module is maintained by the OCA.
.. image:: https://odoo-community.org/logo.png
:alt: Odoo Community Association
:target: https://odoo-community.org
OCA, or the Odoo Community Association, is a nonprofit organization whose
mission is to support the collaborative development of Odoo features and
promote its widespread use.
This module is part of the `OCA/server-tools <https://github.com/OCA/server-tools/tree/13.0/base_import_odoo>`_ project on GitHub.
You are welcome to contribute. To learn how please visit https://odoo-community.org/page/Contribute.

11
base_import_odoo/__manifest__.py
View File

@ -1,14 +1,15 @@
# Copyright 2017-2018 Therp BV <http://therp.nl>
# (c) 2017-2018 Therp BV <http://therp.nl>
# (c) 2020 Hunki Enterprises BV <https://hunki-enterprises.com>
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
{
"name": "Import from Odoo",
"version": "10.0.1.0.0",
"author": "Therp BV,Odoo Community Association (OCA)",
"version": "13.0.1.0.0",
"author": "Therp BV,Hunki Enterprises BV,Odoo Community Association (OCA)",
"license": "AGPL-3",
"category": "Tools",
"summary": "Import records from another Odoo instance",
"website": "https://github.com/OCA/server-tools",
"depends": ["mail",],
"depends": ["mail", "base_sparse_field"],
"demo": [
"demo/res_partner.xml",
"demo/res_users.xml",
@ -24,5 +25,5 @@
"views/menu.xml",
],
"installable": True,
"external_dependencies": {"python": ["odoorpc"],},
"external_dependencies": {"python": ["odoorpc"]},
}

38
base_import_odoo/demo/import_odoo_database_field.xml
View File

@ -1,11 +1,11 @@
<?xml version="1.0" encoding="UTF-8" ?>
<odoo noupdate="1">
<record id="mapping_partner_id_root" model="import.odoo.database.field">
<record id="mapping_partner_id_admin" model="import.odoo.database.field">
<field name="database_id" ref="demodb" />
<field name="mapping_type">fixed</field>
<field name="model_id" ref="base.model_res_partner" />
<field name="local_id" ref="base.partner_root" />
<field name="remote_id" ref="base.partner_root" />
<field name="local_id" ref="base.partner_admin" />
<field name="remote_id" ref="base.partner_admin" />
</record>
<record id="mapping_partner_id_company" model="import.odoo.database.field">
<field name="database_id" ref="demodb" />
@ -14,27 +14,27 @@
<field name="local_id" ref="base.main_partner" />
<field name="remote_id" ref="base.main_partner" />
</record>
<record id="mapping_partner_id_public" model="import.odoo.database.field">
<record id="mapping_partner_id_root" model="import.odoo.database.field">
<field name="database_id" ref="demodb" />
<field name="mapping_type">fixed</field>
<field name="model_id" ref="base.model_res_partner" />
<field name="local_id" ref="base.public_partner" />
<field name="remote_id" ref="base.public_partner" />
<field name="local_id" ref="base.partner_root" />
<field name="remote_id" ref="base.partner_root" />
</record>
<record id="mapping_user_admin" model="import.odoo.database.field">
<field name="database_id" ref="demodb" />
<field name="mapping_type">fixed</field>
<field name="model_id" ref="base.model_res_users" />
<field name="local_id" ref="mapped_admin" />
<field name="remote_id" ref="base.user_admin" />
</record>
<record id="mapping_user_root" model="import.odoo.database.field">
<field name="database_id" ref="demodb" />
<field name="mapping_type">fixed</field>
<field name="model_id" ref="base.model_res_users" />
<field name="local_id" ref="mapped_admin" />
<field name="local_id" ref="base.user_root" />
<field name="remote_id" ref="base.user_root" />
</record>
<record id="mapping_user_public" model="import.odoo.database.field">
<field name="database_id" ref="demodb" />
<field name="mapping_type">fixed</field>
<field name="model_id" ref="base.model_res_users" />
<field name="local_id" ref="base.public_user" />
<field name="remote_id" ref="base.public_user" />
</record>
<record id="mapping_company_id" model="import.odoo.database.field">
<field name="database_id" ref="demodb" />
<field name="mapping_type">fixed</field>
@ -46,25 +46,25 @@
<field name="database_id" ref="demodb" />
<field name="mapping_type">unique</field>
<field name="model_id" ref="base.model_res_users" />
<field name="field_ids" eval="[(4, ref('base.field_res_users_login'))]" />
<field name="field_ids" eval="[(4, ref('base.field_res_users__login'))]" />
</record>
<record id="mapping_models" model="import.odoo.database.field">
<field name="database_id" ref="demodb" />
<field name="mapping_type">by_field</field>
<field name="model_id" ref="base.model_ir_model" />
<field name="field_ids" eval="[(4, ref('base.field_ir_model_name'))]" />
<field name="field_ids" eval="[(4, ref('base.field_ir_model__name'))]" />
</record>
<record id="mapping_groups" model="import.odoo.database.field">
<field name="database_id" ref="demodb" />
<field name="mapping_type">by_field</field>
<field name="model_id" ref="base.model_res_groups" />
<field name="field_ids" eval="[(4, ref('base.field_res_groups_name'))]" />
<field name="field_ids" eval="[(4, ref('base.field_res_groups__name'))]" />
</record>
<record id="mapping_attachment" model="import.odoo.database.field">
<field name="database_id" ref="demodb" />
<field name="mapping_type">by_reference</field>
<field name="model_id" ref="base.model_ir_attachment" />
<field name="model_field_id" ref="base.field_ir_attachment_res_model" />
<field name="id_field_id" ref="base.field_ir_attachment_res_id" />
<field name="model_field_id" ref="base.field_ir_attachment__res_model" />
<field name="id_field_id" ref="base.field_ir_attachment__res_id" />
</record>
</odoo>

67
base_import_odoo/models/import_odoo_database.py
View File

@ -1,9 +1,10 @@
# Copyright 2017-2018 Therp BV <http://therp.nl>
# Copyright 2020 Hunki Enterprises BV <https://hunki-enterprises.com>
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
import logging
import traceback
from collections import namedtuple
from urlparse import urlparse
from urllib.parse import urlparse
import psycopg2
@ -86,27 +87,25 @@ class ImportOdooDatabase(models.Model):
required=True,
)
@api.multi
def action_import(self):
"""Create a cronjob to run the actual import"""
self.ensure_one()
if self.cronjob_id:
return self.cronjob_id.write(
{"numbercall": 1, "doall": True, "active": True,}
{"numbercall": 1, "doall": True, "active": True}
)
return self.write({"cronjob_id": self._create_cronjob().id,})
return self.write({"cronjob_id": self._create_cronjob().id})
@api.model
def _run_import_cron(self, ids):
return self.browse(ids)._run_import()
@api.multi
def _run_import(self, commit=True, commit_threshold=100):
"""Run the import as cronjob, commit often"""
self.ensure_one()
if not self.password:
self.write(
{"status_data": dict(self.status_data, error="No password provided",),}
{"status_data": dict(self.status_data, error="No password provided")}
)
return
# model name: [ids]
@ -160,7 +159,7 @@ class ImportOdooDatabase(models.Model):
commit and (commit_threshold or 1) or len(remote_ids[model._name])
)
for start_index in range(len(remote_ids[model._name]) / chunk_len + 1):
for start_index in range(int(len(remote_ids[model._name]) / chunk_len) + 1):
index = start_index * chunk_len
ids = remote_ids[model._name][index : index + chunk_len]
context = ImportContext(
@ -179,9 +178,7 @@ class ImportOdooDatabase(models.Model):
# pragma: no cover
error = traceback.format_exc()
self.env.cr.rollback()
self.write(
{"status_data": dict(self.status_data, error=error),}
)
self.write({"status_data": dict(self.status_data, error=error)})
# pylint: disable=invalid-commit
self.env.cr.commit()
raise
@ -195,17 +192,14 @@ class ImportOdooDatabase(models.Model):
for dummy_model, remote_id in dummies.keys():
if remote_id:
missing.setdefault(dummy_model, []).append(remote_id)
self.write(
{"status_data": dict(self.status_data, dummies=dict(missing)),}
)
self.write({"status_data": dict(self.status_data, dummies=dict(missing))})
@api.multi
def _run_import_model(self, context):
"""Import records of a configured model"""
model = self.env[context.model_line.model_id.model]
fields = self._run_import_model_get_fields(context)
for data in context.remote.execute(
model._name, "read", context.ids, fields.keys()
model._name, "read", context.ids, list(fields.keys())
):
self._run_import_get_record(
context, model, data, create_dummy=False,
@ -226,11 +220,10 @@ class ImportOdooDatabase(models.Model):
context, model, _id, record.id,
)
@api.multi
def _create_record(self, context, model, record):
"""Create a record, add an xmlid"""
_id = record.pop("id")
xmlid = "%d-%s-%d" % (self.id, model._name.replace(".", "_"), _id or 0,)
xmlid = "%d-%s-%d" % (self.id, model._name.replace(".", "_"), _id or 0)
record = self._create_record_filter_fields(model, record)
model_defaults = {}
if context.model_line.defaults:
@ -264,7 +257,7 @@ class ImportOdooDatabase(models.Model):
return new
def _create_record_xmlid(self, model, local_id, remote_id):
xmlid = "%d-%s-%d" % (self.id, model._name.replace(".", "_"), remote_id or 0,)
xmlid = "%d-%s-%d" % (self.id, model._name.replace(".", "_"), remote_id or 0)
if self.env.ref("base_import_odoo.%s" % xmlid, False):
return
return self.env["ir.model.data"].create(
@ -282,7 +275,7 @@ class ImportOdooDatabase(models.Model):
def _create_record_filter_fields(self, model, record):
"""Return a version of record with unknown fields for model removed
and required fields with no value set to the default if it exists"""
defaults = model.default_get(record.keys())
defaults = model.default_get(list(record.keys()))
return {
key: value
if value or not model._fields[key].required
@ -300,7 +293,6 @@ class ImportOdooDatabase(models.Model):
context["no_reset_password"] = True
return context
@api.multi
def _run_import_get_record(
self, context, model, record, create_dummy=True,
):
@ -359,7 +351,6 @@ class ImportOdooDatabase(models.Model):
)
return _id
@api.multi
def _run_import_get_record_mapping(
self, context, model, record, create_dummy=True,
):
@ -422,7 +413,6 @@ class ImportOdooDatabase(models.Model):
raise exceptions.UserError(_("Unknown mapping"))
return _id
@api.multi
def _run_import_create_dummy(
self, context, model, record, forcecreate=False,
):
@ -512,10 +502,9 @@ class ImportOdooDatabase(models.Model):
)
return dummy.id
@api.multi
def _run_import_map_values(self, context, data):
model = self.env[context.model_line.model_id.model]
for field_name in data.keys():
for field_name in list(data.keys()):
if (
not isinstance(model._fields[field_name], fields._Relational)
or not data[field_name]
@ -547,7 +536,7 @@ class ImportOdooDatabase(models.Model):
)
for _id in ids
]
data[field_name] = filter(None, data[field_name])
data[field_name] = list(filter(None, data[field_name]))
if model._fields[field_name].type == "many2one":
if data[field_name]:
data[field_name] = data[field_name] and data[field_name][0]
@ -563,7 +552,7 @@ class ImportOdooDatabase(models.Model):
value = data.get(field.name, "")
counter = 1
while model.with_context(active_test=False).search(
[(field.name, "=", data.get(field.name, value)),]
[(field.name, "=", data.get(field.name, value))]
):
data[field.name] = "%s (%d)" % (value, counter)
counter += 1
@ -594,7 +583,6 @@ class ImportOdooDatabase(models.Model):
data.update(update)
return data
@api.multi
def _run_import_model_get_fields(self, context):
return {
name: field
@ -604,7 +592,6 @@ class ImportOdooDatabase(models.Model):
if not field.compute or field.inverse
}
@api.multi
def _run_import_model_cleanup_dummies(self, context, model, remote_id, local_id):
if not (model._name, remote_id) in context.dummies:
return
@ -630,7 +617,7 @@ class ImportOdooDatabase(models.Model):
if record._fields[field_name].type == "many2one":
record.write({field_name: local_id})
elif record._fields[field_name].type == "many2many":
record.write({field_name: [(3, dummy_id), (4, local_id),]})
record.write({field_name: [(3, dummy_id), (4, local_id)]})
else:
raise exceptions.UserError(
_("Unhandled field type %s") % record._fields[field_name].type
@ -662,7 +649,6 @@ class ImportOdooDatabase(models.Model):
return remote
@api.constrains("url", "database", "user", "password")
@api.multi
def _constrain_url(self):
for this in self:
if this == self.env.ref("base_import_odoo.demodb", False):
@ -674,19 +660,21 @@ class ImportOdooDatabase(models.Model):
this._get_connection()
@api.depends("status_data")
@api.multi
def _compute_status_html(self):
for this in self:
if not this.status_data:
this.status_html = False
continue
this.status_html = self.env.ref(
"base_import_odoo.view_import_odoo_database_qweb"
).render({"object": this})
this.status_html = (
self.env.ref("base_import_odoo.view_import_odoo_database_qweb")
.render({"object": this})
.decode("utf8")
)
@api.depends("cronjob_id")
@api.multi
def _compute_cronjob_running(self):
for this in self:
this.cronjob_running = False
if not this.cronjob_id:
continue
try:
@ -700,20 +688,19 @@ class ImportOdooDatabase(models.Model):
except psycopg2.OperationalError:
this.cronjob_running = True
@api.multi
def _create_cronjob(self):
self.ensure_one()
return self.env["ir.cron"].create(
{
"name": self.display_name,
"model": self._name,
"function": "_run_import_cron",
"model_id": self.env["ir.model"]
.search([("model", "=", self._name)])
.id,
"code": "model._run_import_cron({})".format(self.ids),
"doall": True,
"args": str((self.ids,)),
}
)
@api.multi
def name_get(self):
return [
(this.id, "{}@{}, {}".format(this.user, this.url, this.database))

15
base_import_odoo/models/import_odoo_database_field.py
View File

@ -1,6 +1,7 @@
# Copyright 2017-2018 Therp BV <http://therp.nl>
# Copyright 2020 Hunki Enterprises BV <https://hunki-enterprises.com>
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
from odoo import api, fields, models
from odoo import fields, models
class ImportOdooDatabaseField(models.Model):
@ -15,7 +16,7 @@ class ImportOdooDatabaseField(models.Model):
model_id = fields.Many2one(
"ir.model", string="Model", required=True, ondelete="cascade",
)
model = fields.Char(related=["model_id", "model"])
model = fields.Char(string="Model name", related=["model_id", "model"])
field_ids = fields.Many2many(
"ir.model.fields",
string="Field",
@ -31,8 +32,12 @@ class ImportOdooDatabaseField(models.Model):
id_field_id = fields.Many2one(
"ir.model.fields",
string="ID field",
compute=lambda self: self._compute_reference_field("id_field_id", "integer"),
inverse=lambda self: self._inverse_reference_field("id_field_id", "integer"),
compute=lambda self: self._compute_reference_field(
"id_field_id", "many2one_reference"
),
inverse=lambda self: self._inverse_reference_field(
"id_field_id", "many2one_reference"
),
)
# TODO: create a reference function field to set this conveniently
local_id = fields.Integer(
@ -58,12 +63,10 @@ class ImportOdooDatabaseField(models.Model):
default="fixed",
)
@api.multi
def _compute_reference_field(self, field_name, ttype):
for this in self:
this[field_name] = this.field_ids.filtered(lambda x: x.ttype == ttype)
@api.multi
def _inverse_reference_field(self, field_name, ttype):
self.field_ids = (
self.field_ids.filtered(lambda x: x.ttype != ttype) + self[field_name]

0
base_import_odoo/CONFIGURE.rst → base_import_odoo/readme/CONFIGURE.rst
View File

1
base_import_odoo/readme/CONTRIBUTORS.rst 100644
View File

@ -0,0 +1 @@
* Holger Brunn <mail@hunki-enterprises.com>

0
base_import_odoo/DESCRIPTION.rst → base_import_odoo/readme/DESCRIPTION.rst
View File

2
base_import_odoo/ROADMAP.rst → base_import_odoo/readme/ROADMAP.rst
View File

@ -1,5 +1,3 @@
* Yes of course this duplicates a lot of connector functionality. Rewrite this with the connector framework, probably collaborate with https://github.com/OCA/connector-odoo2odoo
* Do something with workflows
* Support reference fields, while being at it refactor _run_import_map_values to call a function per field type
* Probably it's safer and faster to disable recomputation during import, and recompute all fields afterwards
* Add duplicate handling strategy 'Overwrite older'

0
base_import_odoo/USAGE.rst → base_import_odoo/readme/USAGE.rst
View File

448
base_import_odoo/static/description/index.html 100644
View File

@ -0,0 +1,448 @@
<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="Docutils 0.15.2: http://docutils.sourceforge.net/" />
<title>Import from Odoo</title>
<style type="text/css">
/*
:Author: David Goodger (goodger@python.org)
:Id: $Id: html4css1.css 7952 2016-07-26 18:15:59Z milde $
:Copyright: This stylesheet has been placed in the public domain.
Default cascading style sheet for the HTML output of Docutils.
See http://docutils.sf.net/docs/howto/html-stylesheets.html for how to
customize this style sheet.
*/
/* used to remove borders from tables and images */
.borderless, table.borderless td, table.borderless th {
border: 0 }
table.borderless td, table.borderless th {
/* Override padding for "table.docutils td" with "! important".
The right padding separates the table cells. */
padding: 0 0.5em 0 0 ! important }
.first {
/* Override more specific margin styles with "! important". */
margin-top: 0 ! important }
.last, .with-subtitle {
margin-bottom: 0 ! important }
.hidden {
display: none }
.subscript {
vertical-align: sub;
font-size: smaller }
.superscript {
vertical-align: super;
font-size: smaller }
a.toc-backref {
text-decoration: none ;
color: black }
blockquote.epigraph {
margin: 2em 5em ; }
dl.docutils dd {
margin-bottom: 0.5em }
object[type="image/svg+xml"], object[type="application/x-shockwave-flash"] {
overflow: hidden;
}
/* Uncomment (and remove this text!) to get bold-faced definition list terms
dl.docutils dt {
font-weight: bold }
*/
div.abstract {
margin: 2em 5em }
div.abstract p.topic-title {
font-weight: bold ;
text-align: center }
div.admonition, div.attention, div.caution, div.danger, div.error,
div.hint, div.important, div.note, div.tip, div.warning {
margin: 2em ;
border: medium outset ;
padding: 1em }
div.admonition p.admonition-title, div.hint p.admonition-title,
div.important p.admonition-title, div.note p.admonition-title,
div.tip p.admonition-title {
font-weight: bold ;
font-family: sans-serif }
div.attention p.admonition-title, div.caution p.admonition-title,
div.danger p.admonition-title, div.error p.admonition-title,
div.warning p.admonition-title, .code .error {
color: red ;
font-weight: bold ;
font-family: sans-serif }
/* Uncomment (and remove this text!) to get reduced vertical space in
compound paragraphs.
div.compound .compound-first, div.compound .compound-middle {
margin-bottom: 0.5em }
div.compound .compound-last, div.compound .compound-middle {
margin-top: 0.5em }
*/
div.dedication {
margin: 2em 5em ;
text-align: center ;
font-style: italic }
div.dedication p.topic-title {
font-weight: bold ;
font-style: normal }
div.figure {
margin-left: 2em ;
margin-right: 2em }
div.footer, div.header {
clear: both;
font-size: smaller }
div.line-block {
display: block ;
margin-top: 1em ;
margin-bottom: 1em }
div.line-block div.line-block {
margin-top: 0 ;
margin-bottom: 0 ;
margin-left: 1.5em }
div.sidebar {
margin: 0 0 0.5em 1em ;
border: medium outset ;
padding: 1em ;
background-color: #ffffee ;
width: 40% ;
float: right ;
clear: right }
div.sidebar p.rubric {
font-family: sans-serif ;
font-size: medium }
div.system-messages {
margin: 5em }
div.system-messages h1 {
color: red }
div.system-message {
border: medium outset ;
padding: 1em }
div.system-message p.system-message-title {
color: red ;
font-weight: bold }
div.topic {
margin: 2em }
h1.section-subtitle, h2.section-subtitle, h3.section-subtitle,
h4.section-subtitle, h5.section-subtitle, h6.section-subtitle {
margin-top: 0.4em }
h1.title {
text-align: center }
h2.subtitle {
text-align: center }
hr.docutils {
width: 75% }
img.align-left, .figure.align-left, object.align-left, table.align-left {
clear: left ;
float: left ;
margin-right: 1em }
img.align-right, .figure.align-right, object.align-right, table.align-right {
clear: right ;
float: right ;
margin-left: 1em }
img.align-center, .figure.align-center, object.align-center {
display: block;
margin-left: auto;
margin-right: auto;
}
table.align-center {
margin-left: auto;
margin-right: auto;
}
.align-left {
text-align: left }
.align-center {
clear: both ;
text-align: center }
.align-right {
text-align: right }
/* reset inner alignment in figures */
div.align-right {
text-align: inherit }
/* div.align-center * { */
/* text-align: left } */
.align-top {
vertical-align: top }
.align-middle {
vertical-align: middle }
.align-bottom {
vertical-align: bottom }
ol.simple, ul.simple {
margin-bottom: 1em }
ol.arabic {
list-style: decimal }
ol.loweralpha {
list-style: lower-alpha }
ol.upperalpha {
list-style: upper-alpha }
ol.lowerroman {
list-style: lower-roman }
ol.upperroman {
list-style: upper-roman }
p.attribution {
text-align: right ;
margin-left: 50% }
p.caption {
font-style: italic }
p.credits {
font-style: italic ;
font-size: smaller }
p.label {
white-space: nowrap }
p.rubric {
font-weight: bold ;
font-size: larger ;
color: maroon ;
text-align: center }
p.sidebar-title {
font-family: sans-serif ;
font-weight: bold ;
font-size: larger }
p.sidebar-subtitle {
font-family: sans-serif ;
font-weight: bold }
p.topic-title {
font-weight: bold }
pre.address {
margin-bottom: 0 ;
margin-top: 0 ;
font: inherit }
pre.literal-block, pre.doctest-block, pre.math, pre.code {
margin-left: 2em ;
margin-right: 2em }
pre.code .ln { color: grey; } /* line numbers */
pre.code, code { background-color: #eeeeee }
pre.code .comment, code .comment { color: #5C6576 }
pre.code .keyword, code .keyword { color: #3B0D06; font-weight: bold }
pre.code .literal.string, code .literal.string { color: #0C5404 }
pre.code .name.builtin, code .name.builtin { color: #352B84 }
pre.code .deleted, code .deleted { background-color: #DEB0A1}
pre.code .inserted, code .inserted { background-color: #A3D289}
span.classifier {
font-family: sans-serif ;
font-style: oblique }
span.classifier-delimiter {
font-family: sans-serif ;
font-weight: bold }
span.interpreted {
font-family: sans-serif }
span.option {
white-space: nowrap }
span.pre {
white-space: pre }
span.problematic {
color: red }
span.section-subtitle {
/* font-size relative to parent (h1..h6 element) */
font-size: 80% }
table.citation {
border-left: solid 1px gray;
margin-left: 1px }
table.docinfo {
margin: 2em 4em }
table.docutils {
margin-top: 0.5em ;
margin-bottom: 0.5em }
table.footnote {
border-left: solid 1px black;
margin-left: 1px }
table.docutils td, table.docutils th,
table.docinfo td, table.docinfo th {
padding-left: 0.5em ;
padding-right: 0.5em ;
vertical-align: top }
table.docutils th.field-name, table.docinfo th.docinfo-name {
font-weight: bold ;
text-align: left ;
white-space: nowrap ;
padding-left: 0 }
/* "booktabs" style (no vertical lines) */
table.docutils.booktabs {
border: 0px;
border-top: 2px solid;
border-bottom: 2px solid;
border-collapse: collapse;
}
table.docutils.booktabs * {
border: 0px;
}
table.docutils.booktabs th {
border-bottom: thin solid;
text-align: left;
}
h1 tt.docutils, h2 tt.docutils, h3 tt.docutils,
h4 tt.docutils, h5 tt.docutils, h6 tt.docutils {
font-size: 100% }
ul.auto-toc {
list-style-type: none }
</style>
</head>
<body>
<div class="document" id="import-from-odoo">
<h1 class="title">Import from Odoo</h1>
<!-- !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!! This file is generated by oca-gen-addon-readme !!
!! changes will be overwritten. !!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! -->
<p><a class="reference external" href="https://odoo-community.org/page/development-status"><img alt="Beta" src="https://img.shields.io/badge/maturity-Beta-yellow.png" /></a> <a class="reference external" href="http://www.gnu.org/licenses/agpl-3.0-standalone.html"><img alt="License: AGPL-3" src="https://img.shields.io/badge/licence-AGPL--3-blue.png" /></a> <a class="reference external" href="https://github.com/OCA/server-tools/tree/13.0/base_import_odoo"><img alt="OCA/server-tools" src="https://img.shields.io/badge/github-OCA%2Fserver--tools-lightgray.png?logo=github" /></a> <a class="reference external" href="https://translation.odoo-community.org/projects/server-tools-13-0/server-tools-13-0-base_import_odoo"><img alt="Translate me on Weblate" src="https://img.shields.io/badge/weblate-Translate%20me-F47D42.png" /></a> <a class="reference external" href="https://runbot.odoo-community.org/runbot/149/13.0"><img alt="Try me on Runbot" src="https://img.shields.io/badge/runbot-Try%20me-875A7B.png" /></a></p>
<p>This module was written to import data from another Odoo database. The idea is that you define which models to import from the other database, and add eventual mappings for records you don’t want to import.</p>
<div class="section" id="use-cases">
<h1>Use cases</h1>
<ul class="simple">
<li>merging databases</li>
<li>one way sync</li>
<li>aggregating management data from distributed systems</li>
</ul>
<p><strong>Table of contents</strong></p>
</div>
<div class="section" id="configuration">
<h1>Configuration</h1>
<p>Go to Settings / Remote Odoo import / Import configurations and create a configuration.</p>
<p>After filling in your credentials, select models you want to import from the remote database. If you only want to import a subset of the records, add an appropriate domain.</p>
<p>The import will copy records of all models listed, and handle links to records of models which are not imported depending on the existing field mappings. Field mappings to local records also are a stopping condition. Without those, the import will have to create some record for all required x2x fields, which you probably don’t want.</p>
<p>Probably you’ll want to map records of model <tt class="docutils literal">res.company</tt>, and at least the admin user.</p>
<p>The module doesn’t import one2many fields, if you want to have those, add the model the field in question points to to the list of imported models, possibly with a domain.</p>
<p>If you don’t fill in a remote ID, the addon will use the configured local ID for every record of the model (this way, you can for example map all users in the remote system to some import user in the current system).</p>
<p>For fields that have a uniqueness constraint (like <tt class="docutils literal">res.users#login</tt>), create a field mapping if type <tt class="docutils literal">unique</tt>, then the import will generate a unique value for this field.</p>
<p>For models using references with two fields (like <tt class="docutils literal">ir.attachment</tt>), create a field mapping of type <tt class="docutils literal">by reference</tt> and select the two fields involved. The import will transform known ids (=ids of models you import) to the respective local id, and clean out the model/id fields for unknown models/ids.</p>
<p>You can add custom defaults per model in case your database has different required fields than the source database. For <tt class="docutils literal">res.partner</tt>, you’ll most certainly fill in <tt class="docutils literal">{'name': <span class="pre">'/'}</span></tt> or somethign similar.</p>
</div>
<div class="section" id="usage">
<h1>Usage</h1>
<p>To use this module, you need to:</p>
<ol class="arabic simple">
<li>go to an import configuration and hit the button <tt class="docutils literal">Run import</tt></li>
<li>be patient, this creates a cronjob which will start up to a minutes afterwards</li>
<li>reload the form, as soon as the cronjob runs you’ll see a field <tt class="docutils literal">Progress</tt> that lets you inspect what was imported already</li>
<li>note that the cronjob also resets the password as soon as it has read it. So for a subsequent import, you’ll have to fill it in again</li>
<li>running an import a second time won’t duplicate data, it should recognize records imported earlier and just update them</li>
</ol>
</div>
<div class="section" id="known-issues-roadmap">
<h1>Known issues / Roadmap</h1>
<ul class="simple">
<li>Yes of course this duplicates a lot of connector functionality. Rewrite this with the connector framework, probably collaborate with <a class="reference external" href="https://github.com/OCA/connector-odoo2odoo">https://github.com/OCA/connector-odoo2odoo</a></li>
<li>Support reference fields, while being at it refactor _run_import_map_values to call a function per field type</li>
<li>Add duplicate handling strategy ‘Overwrite older’</li>
</ul>
</div>
<div class="section" id="bug-tracker">
<h1>Bug Tracker</h1>
<p>Bugs are tracked on <a class="reference external" href="https://github.com/OCA/server-tools/issues">GitHub Issues</a>.
In case of trouble, please check there if your issue has already been reported.
If you spotted it first, help us smashing it by providing a detailed and welcomed
<a class="reference external" href="https://github.com/OCA/server-tools/issues/new?body=module:%20base_import_odoo%0Aversion:%2013.0%0A%0A**Steps%20to%20reproduce**%0A-%20...%0A%0A**Current%20behavior**%0A%0A**Expected%20behavior**">feedback</a>.</p>
<p>Do not contact contributors directly about support or help with technical issues.</p>
</div>
<div class="section" id="credits">
<h1>Credits</h1>
<div class="section" id="authors">
<h2>Authors</h2>
<ul class="simple">
<li>Therp BV</li>
<li>Hunki Enterprises BV</li>
</ul>
</div>
<div class="section" id="contributors">
<h2>Contributors</h2>
<ul class="simple">
<li>Holger Brunn &lt;<a class="reference external" href="mailto:mail&#64;hunki-enterprises.com">mail&#64;hunki-enterprises.com</a>&gt;</li>
</ul>
</div>
<div class="section" id="maintainers">
<h2>Maintainers</h2>
<p>This module is maintained by the OCA.</p>
<a class="reference external image-reference" href="https://odoo-community.org"><img alt="Odoo Community Association" src="https://odoo-community.org/logo.png" /></a>
<p>OCA, or the Odoo Community Association, is a nonprofit organization whose
mission is to support the collaborative development of Odoo features and
promote its widespread use.</p>
<p>This module is part of the <a class="reference external" href="https://github.com/OCA/server-tools/tree/13.0/base_import_odoo">OCA/server-tools</a> project on GitHub.</p>
<p>You are welcome to contribute. To learn how please visit <a class="reference external" href="https://odoo-community.org/page/Contribute">https://odoo-community.org/page/Contribute</a>.</p>
</div>
</div>
</div>
</body>
</html>

9
base_import_odoo/tests/test_base_import_odoo.py
View File

@ -1,4 +1,5 @@
# Copyright 2017-2018 Therp BV <http://therp.nl>
# Copyright 2020 Hunki Enterprises BV <https://hunki-enterprises.com>
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
from mock import patch
@ -53,11 +54,9 @@ class TestBaseImportOdoo(TransactionCase):
group_count = self.env["res.groups"].search([], count=True)
user_count = self.env["res.users"].search([], count=True)
run = 1
for dummy in range(2):
for _dummy in range(2):
# we run this two times to enter the code path where xmlids exist
self.env.ref("base_import_odoo.demodb").write(
{"password": "admin",}
)
self.env.ref("base_import_odoo.demodb").write({"password": "admin"})
with patch("odoorpc.ODOO.execute", side_effect=_mock_execute):
self.env.ref("base_import_odoo.demodb")._run_import()
# here the actual test begins - check that we created new
@ -88,7 +87,7 @@ class TestBaseImportOdoo(TransactionCase):
# check that there's a new attachment
attachment = self.env.ref("base_import_odoo.attachment_demo")
imported_attachment = self.env["ir.attachment"].search(
[("res_model", "=", "res.users"), ("res_id", "=", imported_user.id),]
[("res_model", "=", "res.users"), ("res_id", "=", imported_user.id)]
)
self.assertTrue(attachment)
self.assertEqual(attachment.datas, imported_attachment.datas)

47
base_import_odoo/views/import_odoo_database.xml
View File

@ -87,27 +87,32 @@
</record>
<template id="view_import_odoo_database_qweb">
<script type="text/javascript">
function base_import_database_open(model, model_name, database_id)
{
return new openerp.web.Model('ir.model.data')
.query(['res_id'])
.filter([
['module', '=', 'base_import_odoo'],
['model', '=', model],
['import_database_id', '=', database_id],
])
.all()
.then(function(data)
{
return openerp.webclient.action_manager.do_action({
'name': model_name,
'type': 'ir.actions.act_window',
'views': [[false, 'list'], [false, 'form']],
'res_model': model,
'domain': [['id', 'in', _.map(data, function(x) {return x.res_id})]]
odoo.define('base_import_odoo', function(require) {
return function base_import_database_open(model, model_name, database_id) {
return require('web.rpc')
.query({
model: 'ir.model.data',
method: 'search_read',
fields: ['res_id'],
domain: [
['module', '=', 'base_import_odoo'],
['model', '=', model],
['import_database_id', '=', database_id],
],
})
.then(function(data)
{
debugger;
return require('web.web_client').action_manager.do_action({
'name': model_name,
'type': 'ir.actions.act_window',
'views': [[false, 'list'], [false, 'form']],
'res_model': model,
'domain': [['id', 'in', _.map(data, function(x) {return x.res_id})]]
});
});
});
}
}
})
</script>
<h2 t-if="object.cronjob_running">Import progress</h2>
<h2 t-if="not object.cronjob_running">Import results</h2>
@ -122,7 +127,7 @@
<h3 t-esc="model_display_name" />
<a
href="#"
t-att-onclick="'base_import_database_open(&quot;%s&quot;, &quot;%s&quot;, %s)' % (model_name, model_display_name, object.id)"
t-att-onclick="'odoo.define(function(require) {require(&quot;base_import_odoo&quot;)(&quot;%s&quot;, &quot;%s&quot;, %s)})' % (model_name, model_display_name, object.id)"
>
<span
t-esc="object.status_data.get('done', {}).get(model_name, 0)"