search

OCA / OpenUpgrade

Open source upgrade path for Odoo/OpenERP
https://oca.github.io/OpenUpgrade/
GNU Affero General Public License v3.0
726 stars 690 forks source link

Migration Step for Odoo v15 to v16 with docker #3846

Closed ikus060 closed 1 year ago

ikus060 commented 1 year ago

Hi there!

I'm trying to upgrade my Odoo instance from version 15 to version 16 using the Odoo OpenUpgrade tool. However, when running Odoo in a container with the Docker image "odoo:15.0", I can't seem to find the migration step for this version upgrade.

I've looked through the documentation and searched online, but haven't been able to find any information on this specific issue. Can someone please help me figure out how to access the migration step for upgrading from Odoo v15 to v16 when running Odoo in a container with the "odoo:15.0" Docker image?

legalsylvain commented 1 year ago

Hi, thanks for your interest in Openupgrade project.

When you want to upgrade from 15.0 to 16.0 you have to use odoo 16.0 + openupgrade 16.0. This may not be very clear in the documentation. As I am currently working on it, I will try to clarify this important point.

Once you use the correct version, you have to follow the instruction and set the openupgrade modules in the addons_path.

ikus060 commented 1 year ago

When you want to upgrade from 15.0 to 16.0 you have to use odoo 16.0 + openupgrade 16.0.

While it make sens, I have no idea what is the implication of this statement. Do I need to start an additional container with odoo:16.0

I search for a docker image named "openupgrade" in dockerhub and did not found any. At least none of them looked official.

What would be helpful is an example of how odoo it running then provide the steps with command line to upgrade ?

e.g.: I'm running odoo with the following docker-compose:

version: "3.1"
services:
  web:
    image: odoo:15.0
    restart: "always"
    deploy:
      update_config:
        parallelism: 1
        delay: 30s
        order: start-first
    depends_on:
      - db
    volumes:
      - ./data/addons/:/mnt/extra-addons
      - ./data/conf/:/etc/odoo/
      - ./data/data/:/var/lib/odoo
    environment:
      ODOO_DATABASE_HOST: db
      ODOO_DATABASE_USER: odoo
      ODOO_DATABASE_PASSWORD: odoo
      ODOO_DATABASE_NAME: postgres
    ports:
      - 8069:8069
  db:
    image: postgres:13
    restart: "always"
    environment:
      - POSTGRES_DB=postgres
      - POSTGRES_PASSWORD=odoo
      - POSTGRES_USER=odoo
    volumes:
      - ./db:/var/lib/postgresql/data
legalsylvain commented 1 year ago

While it make sens, I have no idea what is the implication of this statement. Do I need to start an additional container with odoo:16.0

yes.

I search for a docker image named "openupgrade" in dockerhub and did not found any. At least none of them looked official.

There is no docker image named openupgrade. openupgrade is two modules that have to be present in your addons path.

What would be helpful is an example of how odoo it running then provide the steps with command line to upgrade ?

that seems out of the topic of OpenUpgrade. Using openupgrade requires to know how to use odoo, use extra addons, merge various PR and make an update of the database. These are the prerequisites. Same here, I'll try to make the documentation more explicit on that topic. I advise you to read up on how odoo works, or to turn to an odoo integrator.

regards.

ikus060 commented 1 year ago

I'm still trying to stick the puzzle together.

How am I suppose to install "openupgradelib" within the Docker container ? I try adding it to the addons folder, but it doesn't seams to get pick-up by odoo. I get the following exception:

2023-05-03 16:29:00,317 1 INFO ? odoo: Odoo version 16.0-20230430 
2023-05-03 16:29:00,317 1 INFO ? odoo: Using configuration file at /etc/odoo/odoo.conf 
2023-05-03 16:29:00,317 1 INFO ? odoo: addons paths: ['/usr/lib/python3/dist-packages/odoo/addons', '/var/lib/odoo/addons/16.0', '/mnt/extra-addons'] 
2023-05-03 16:29:00,317 1 INFO ? odoo: database: odoo@db:5432 
2023-05-03 16:29:00,392 1 INFO ? odoo.addons.base.models.ir_actions_report: Will use the Wkhtmltopdf binary at /usr/local/bin/wkhtmltopdf 
2023-05-03 16:29:00,532 1 CRITICAL ? odoo.modules.module: Couldn't load module openupgrade_framework 
2023-05-03 16:29:00,532 1 CRITICAL ? odoo.modules.module: No module named 'openupgradelib' 
2023-05-03 16:29:00,532 1 ERROR ? odoo.service.server: Failed to load server-wide module `openupgrade_framework`. 
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/odoo/service/server.py", line 1248, in load_server_wide_modules
    odoo.modules.module.load_openerp_module(m)
  File "/usr/lib/python3/dist-packages/odoo/modules/module.py", line 432, in load_openerp_module
    __import__('odoo.addons.' + module_name)
  File "/mnt/extra-addons/openupgrade_framework/__init__.py", line 7, in <module>
    from . import odoo_patch
  File "/mnt/extra-addons/openupgrade_framework/odoo_patch/__init__.py", line 1, in <module>
    from . import odoo
  File "/mnt/extra-addons/openupgrade_framework/odoo_patch/odoo/__init__.py", line 1, in <module>
    from . import addons, api, models, modules, tests
  File "/mnt/extra-addons/openupgrade_framework/odoo_patch/odoo/addons/__init__.py", line 1, in <module>
    from . import base
  File "/mnt/extra-addons/openupgrade_framework/odoo_patch/odoo/addons/base/__init__.py", line 1, in <module>
    from . import models
  File "/mnt/extra-addons/openupgrade_framework/odoo_patch/odoo/addons/base/models/__init__.py", line 1, in <module>
    from . import ir_model
  File "/mnt/extra-addons/openupgrade_framework/odoo_patch/odoo/addons/base/models/ir_model.py", line 3, in <module>
    from openupgradelib import openupgrade
ModuleNotFoundError: No module named 'openupgradelib'
2023-05-03 16:29:00,533 1 INFO ? odoo.service.server: Initiating shutdown 
2023-05-03 16:29:00,533 1 INFO ? odoo.service.server: Hit CTRL-C again or send a second signal to force the shutdown.
legalsylvain commented 1 year ago

Hi Sorry, the question about how to install a python library in a docker container is out of the scope of the openupgrade project. Please consult docker documentation.

Kind regards.

ikus060 commented 1 year ago

Maybe we should clarify that OCA/OpenUpgrade doesn't support Docker deployment ?

So far, I came up with the following procedure to initiate the upgrade. but even then, I feel alot of information is missing in the documentation.

During the upgrade, I saw a couple of SQL exceptions:

db_1   | 2023-05-03 16:56:13.169 UTC [42] ERROR:  could not create unique index "utm_source_unique_name"
db_1   | 2023-05-03 16:56:13.169 UTC [42] DETAIL:  Key (name)=(rdiffweb.org : where to find everything about the simplified backup management software 2022-06-13 12:40:55 (copie) (copie) (copie)) is duplicated.
db_1   | 2023-05-03 16:56:13.169 UTC [42] STATEMENT:  ALTER TABLE "utm_source" ADD CONSTRAINT "utm_source_unique_name" UNIQUE(name)
db_1   | 2023-05-03 16:56:13.907 UTC [42] ERROR:  invalid input syntax for type timestamp: ""
db_1   | 2023-05-03 16:56:13.907 UTC [42] STATEMENT:  
db_1   |            ALTER TABLE "mail_mail"
db_1   |            ALTER COLUMN "scheduled_date" DROP DEFAULT,
db_1   |            ALTER COLUMN "scheduled_date" TYPE timestamp USING "scheduled_date"::timestamp

And python is trowing many exception too:

2023-05-03 16:58:46,219 1 INFO ikus odoo.modules.loading: loading 1 modules... 
2023-05-03 16:58:46,224 1 INFO ikus odoo.modules.loading: 1 modules loaded in 0.01s, 0 queries (+0 extra) 
2023-05-03 16:58:46,242 1 INFO ikus odoo.modules.graph: module account_invoice_pricelist_sale: Unmet dependencies: sale 
2023-05-03 16:58:46,242 1 INFO ikus odoo.modules.graph: module sale_sms: Unmet dependencies: sale 
2023-05-03 16:58:46,242 1 INFO ikus odoo.modules.graph: module sale: Unmet dependencies: account_payment 
2023-05-03 16:58:46,242 1 INFO ikus odoo.modules.graph: module website_payment: Unmet dependencies: account_payment 
2023-05-03 16:58:46,242 1 INFO ikus odoo.modules.graph: module mass_mailing_sale: Unmet dependencies: sale 
2023-05-03 16:58:46,242 1 INFO ikus odoo.modules.graph: module sale_management: Unmet dependencies: sale 
2023-05-03 16:58:46,243 1 INFO ikus odoo.modules.loading: loading 72 modules... 
2023-05-03 16:58:46,497 1 WARNING ikus odoo.api.create: The model odoo.addons.account_invoice_overdue_reminder.wizard.overdue_reminder_wizard is not overriding the create method in batch 
2023-05-03 16:58:46,587 1 WARNING ikus odoo.api.create: The model odoo.addons.membership_variable_period.models.product_template is not overriding the create method in batch 
2023-05-03 16:58:46,592 1 INFO ikus odoo.modules.loading: 72 modules loaded in 0.35s, 0 queries (+0 extra) 
2023-05-03 16:58:46,691 1 ERROR ikus odoo.modules.loading: Some modules are not loaded, some dependencies or manifest may be missing: ['account_invoice_pricelist_sale', 'mass_mailing_sale', 'sale', 'sale_management', 'sale_sms', 'website_payment'] 
2023-05-03 16:58:46,693 1 INFO ikus odoo.modules.loading: Modules loaded. 
2023-05-03 16:58:46,704 1 ERROR ikus odoo.sql_db: bad query: SELECT "ir_act_server"."id" AS "id", "ir_act_server"."name"->>'en_US' AS "name", "ir_act_server"."type" AS "type", "ir_act_server"."help"->>'en_US' AS "help", "ir_act_server"."binding_model_id" AS "binding_model_id", "ir_act_server"."binding_type" AS "binding_type", "ir_act_server"."binding_view_types" AS "binding_view_types", "ir_act_server"."create_uid" AS "create_uid", "ir_act_server"."create_date" AS "create_date", "ir_act_server"."write_uid" AS "write_uid", "ir_act_server"."write_date" AS "write_date", "ir_act_server"."usage" AS "usage", "ir_act_server"."state" AS "state", "ir_act_server"."sequence" AS "sequence", "ir_act_server"."model_id" AS "model_id", "ir_act_server"."model_name" AS "model_name", "ir_act_server"."code" AS "code", "ir_act_server"."crud_model_id" AS "crud_model_id", "ir_act_server"."link_field_id" AS "link_field_id", "ir_act_server"."template_id" AS "template_id", "ir_act_server"."mail_post_autofollow" AS "mail_post_autofollow", "ir_act_server"."mail_post_method" AS "mail_post_method", "ir_act_server"."activity_type_id" AS "activity_type_id", "ir_act_server"."activity_summary" AS "activity_summary", "ir_act_server"."activity_note" AS "activity_note", "ir_act_server"."activity_date_deadline_range" AS "activity_date_deadline_range", "ir_act_server"."activity_date_deadline_range_type" AS "activity_date_deadline_range_type", "ir_act_server"."activity_user_type" AS "activity_user_type", "ir_act_server"."activity_user_id" AS "activity_user_id", "ir_act_server"."activity_user_field_name" AS "activity_user_field_name", "ir_act_server"."sms_template_id" AS "sms_template_id", "ir_act_server"."sms_method" AS "sms_method", "ir_act_server"."website_path" AS "website_path", "ir_act_server"."website_published" AS "website_published" FROM "ir_act_server" WHERE "ir_act_server".id IN (406, 405, 450, 451)
ERROR: column ir_act_server.mail_post_autofollow does not exist
LINE 1: ..., "ir_act_server"."template_id" AS "template_id", "ir_act_se...
                                                             ^

2023-05-03 16:58:46,706 1 WARNING ikus odoo.modules.loading: Transient module states were reset 
2023-05-03 16:58:46,706 1 ERROR ikus odoo.modules.registry: Failed to load registry 
2023-05-03 16:58:46,707 1 WARNING ikus py.warnings: /usr/lib/python3/dist-packages/odoo/http.py:890: DeprecationWarning: request.session.geoip have been moved to request.geoip
  File "/usr/lib/python3.9/threading.py", line 912, in _bootstrap
    self._bootstrap_inner()
  File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.9/threading.py", line 892, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.9/socketserver.py", line 650, in process_request_thread
    self.finish_request(request, client_address)
  File "/usr/lib/python3.9/socketserver.py", line 360, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib/python3.9/socketserver.py", line 720, in __init__
    self.handle()
  File "/usr/lib/python3/dist-packages/werkzeug/serving.py", line 345, in handle
    BaseHTTPRequestHandler.handle(self)
  File "/usr/lib/python3.9/http/server.py", line 427, in handle
    self.handle_one_request()
  File "/usr/lib/python3/dist-packages/werkzeug/serving.py", line 379, in handle_one_request
    return self.run_wsgi()
  File "/usr/lib/python3/dist-packages/werkzeug/serving.py", line 323, in run_wsgi
    execute(self.server.app)
  File "/usr/lib/python3/dist-packages/werkzeug/serving.py", line 312, in execute
    application_iter = app(environ, start_response)
  File "/usr/lib/python3/dist-packages/odoo/http.py", line 1987, in __call__
    response = request._serve_db()
  File "/usr/lib/python3/dist-packages/odoo/http.py", line 1572, in _serve_db
    root.session_store.save(self.session)
  File "/usr/lib/python3/dist-packages/odoo/http.py", line 838, in save
    super().save(session)
  File "/usr/lib/python3/dist-packages/odoo/tools/_vendor/sessions.py", line 196, in save
    pickle.dump(dict(session), f, pickle.HIGHEST_PROTOCOL)
  File "/usr/lib/python3/dist-packages/odoo/http.py", line 890, in __getitem__
    warnings.warn('request.session.geoip have been moved to request.geoip', DeprecationWarning)

2023-05-03 16:58:46,708 1 ERROR ikus odoo.http: Exception during request handling. 
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/odoo/modules/registry.py", line 64, in __new__
    return cls.registries[db_name]
  File "<decorator-gen-6>", line 2, in __getitem__
  File "/usr/lib/python3/dist-packages/odoo/tools/func.py", line 87, in locked
    return func(inst, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/odoo/tools/lru.py", line 34, in __getitem__
    a = self.d[obj]
KeyError: 'ikus'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/odoo/api.py", line 983, in get
    cache_value = field_cache[record._ids[0]]
KeyError: 2

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/odoo/fields.py", line 1160, in __get__
    value = env.cache.get(record, self)
  File "/usr/lib/python3/dist-packages/odoo/api.py", line 990, in get
    raise CacheMiss(record, field)
odoo.exceptions.CacheMiss: 'base.automation(2,).model_name'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/odoo/api.py", line 983, in get
    cache_value = field_cache[record._ids[0]]
KeyError: 406

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/odoo/fields.py", line 1160, in __get__
    value = env.cache.get(record, self)
  File "/usr/lib/python3/dist-packages/odoo/api.py", line 990, in get
    raise CacheMiss(record, field)
odoo.exceptions.CacheMiss: 'ir.actions.server(406,).model_name'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/odoo/http.py", line 1987, in __call__
    response = request._serve_db()
  File "/usr/lib/python3/dist-packages/odoo/http.py", line 1562, in _serve_db
    self.registry = Registry(self.db).check_signaling()
  File "/usr/lib/python3/dist-packages/odoo/modules/registry.py", line 66, in __new__
    return cls.new(db_name)
  File "<decorator-gen-14>", line 2, in new
  File "/usr/lib/python3/dist-packages/odoo/tools/func.py", line 87, in locked
    return func(inst, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/odoo/modules/registry.py", line 90, in new
    odoo.modules.load_modules(registry, force_demo, status, update_module)
  File "/usr/lib/python3/dist-packages/odoo/modules/loading.py", line 617, in load_modules
    model._register_hook()
  File "/usr/lib/python3/dist-packages/odoo/addons/base_automation/models/base_automation.py", line 510, in _register_hook
    Model = self.env.get(action_rule.model_name)
  File "/usr/lib/python3/dist-packages/odoo/fields.py", line 1209, in __get__
    self.compute_value(recs)
  File "/usr/lib/python3/dist-packages/odoo/fields.py", line 1368, in compute_value
    records._compute_field_value(self)
  File "/usr/lib/python3/dist-packages/odoo/models.py", line 4193, in _compute_field_value
    fields.determine(field.compute, self)
  File "/usr/lib/python3/dist-packages/odoo/fields.py", line 100, in determine
    return needle(records, *args)
  File "/usr/lib/python3/dist-packages/odoo/fields.py", line 701, in _compute_related
    record[self.name] = self._process_related(value[self.related_field.name])
  File "/usr/lib/python3/dist-packages/odoo/models.py", line 5858, in __getitem__
    return self._fields[key].__get__(self, type(self))
  File "/usr/lib/python3/dist-packages/odoo/fields.py", line 1186, in __get__
    recs._fetch_field(self)
  File "/usr/lib/python3/dist-packages/odoo/models.py", line 3162, in _fetch_field
    self._read(fnames)
  File "/usr/lib/python3/dist-packages/odoo/models.py", line 3239, in _read
    cr.execute(query_str, params + [sub_ids])
  File "/usr/lib/python3/dist-packages/odoo/sql_db.py", line 313, in execute
    res = self._obj.execute(query, params)
psycopg2.errors.UndefinedColumn: column ir_act_server.mail_post_autofollow does not exist
LINE 1: ..., "ir_act_server"."template_id" AS "template_id", "ir_act_se...
                                                             ^

2023-05-03 16:58:46,710 1 INFO ikus werkzeug: 172.20.0.1 - - [03/May/2023 16:58:46] "GET /web HTTP/1.1" 500 - 21 0.027 0.496

Here the step I followed:

Upgrade postgresql version

When upgrading Odoo, you may need to also upgrade your postresql database to the latest version.

docker-compose exec {service_name} pg_dumpall -U {postgres_user} > dump.sql

That will create a backup of your database as SQL file. You will need that file to restore your data into the newer postgresql.

Create a new volume or folder for the new postgresql database and edit your compose file to make use of this new volume or folder.

e.g.:

image: postgres:13
...
volumes:
  - ./db:/var/lib/postgresql/data

Should become

image: postgres:15
...
volumes:
  - ./db2:/var/lib/postgresql/data

NOTICE: make sure to add POSTGRES_HOST_AUTH_METHOD=md5 to you docker compose if you are migrating from postgresql:13 to postgresql:15 for backward compatibility.

Restart your docker compose.

docker-compose down
docker-compose up

Execute the following commands to restore your database into the new postgresql instance

docker-compose start
docker exec -it {service_name} bash
psql -U {postgres_user} -d {default_postgres_database} < {mapped_volume_folder_path}/dump.sql

Finnaly, restart your docker compose again, and Odoo should be running with the new database.

Download OpenUpgrade to Odoo's Addons folder

cd ./data/addons
wget https://github.com/OCA/OpenUpgrade/archive/refs/heads/16.0.tar.gz
tar --strip-components=1 -zxvf 16.0.tar.gz OpenUpgrade-16.0/openupgrade_scripts OpenUpgrade-16.0/openupgrade_framework

wget https://github.com/OCA/openupgradelib/archive/refs/heads/master.tar.gz
tar --strip-components=1 -zxvf master.tar.gz openupgradelib-master/openupgradelib

Make sure the adjust the permissions of the folder when using Docker

chown -R 101:101 .

Update Odoo version and change command line in docker compose

Before restarting Odoo, update your docker compose file to the targeted version.

e.g.:

image: odoo:15.0

Should become 

image: odoo:16.0
command: odoo --update all --load=base,web,openupgradelib,openupgrade_framework

Restart odoo

When using docker-compose

docker-compose restart

If you are using plain docker command line:

docker restart [your container name]

Initiate upgrade

Initiate the upgrade by browsing to odoo url.

http://localhost:8069

The log should display multiple SQL statements to upgrade your existing database.

ikus060 commented 1 year ago

I think I manage to narrow the issue the the following problem that occurred during the migration:

web_1      | 2023-05-03 18:56:40,358 1 INFO ikus odoo.modules.loading: Loading module mail (17/72) 
web_1      | 2023-05-03 18:56:40,458 1 INFO ikus odoo.modules.registry: module mail: creating or updating database tables 
db_1       | 2023-05-03 18:56:40.649 UTC [37] ERROR:  invalid input syntax for type timestamp: ""
db_1       | 2023-05-03 18:56:40.649 UTC [37] STATEMENT:  
db_1       |            ALTER TABLE "mail_mail"
db_1       |            ALTER COLUMN "scheduled_date" DROP DEFAULT,
db_1       |            ALTER COLUMN "scheduled_date" TYPE timestamp USING "scheduled_date"::timestamp

It seams to prevent the upgrade from going further.

ikus060 commented 1 year ago

@legalsylvain Then, I manage to get it fixed with the following SQL statement. Maybe it should be added to the migration script ?

    UPDATE public.mail_mail SET scheduled_date = NULL WHERE scheduled_date = '';

But next, I'm running into another issue:

web_1      | 2023-05-03 19:14:34,297 1 INFO ikus odoo.modules.loading: Loading module payment (43/72) 
web_1      | 2023-05-03 19:14:34,444 1 INFO ikus odoo.modules.registry: module payment: creating or updating database tables 
web_1      | 2023-05-03 19:14:34,598 1 INFO ikus odoo.schema: Keep unexpected index res_partner_message_main_attachment_id_index on table res_partner 
web_1      | 2023-05-03 19:14:34,598 1 INFO ikus odoo.schema: Keep unexpected index res_users_create_date_index on table res_users 
web_1      | 2023-05-03 19:14:34,706 1 INFO ikus odoo.modules.loading: loading payment/data/payment_icon_data.xml 
web_1      | 2023-05-03 19:14:34,987 1 INFO ikus odoo.modules.loading: loading payment/data/payment_provider_data.xml 
web_1      | 2023-05-03 19:14:35,003 1 WARNING ikus odoo.modules.loading: Transient module states were reset 
web_1      | 2023-05-03 19:14:35,003 1 ERROR ikus odoo.modules.registry: Failed to load registry 
web_1      | 2023-05-03 19:14:35,003 1 ERROR ikus odoo.http: Exception during request handling. 
web_1      | Traceback (most recent call last):
web_1      |   File "/usr/lib/python3/dist-packages/odoo/modules/registry.py", line 64, in __new__
web_1      |     return cls.registries[db_name]
web_1      |   File "<decorator-gen-6>", line 2, in __getitem__
web_1      |   File "/usr/lib/python3/dist-packages/odoo/tools/func.py", line 87, in locked
web_1      |     return func(inst, *args, **kwargs)
web_1      |   File "/usr/lib/python3/dist-packages/odoo/tools/lru.py", line 34, in __getitem__
web_1      |     a = self.d[obj]
web_1      | KeyError: 'ikus'
web_1      | 
web_1      | During handling of the above exception, another exception occurred:
web_1      | 
web_1      | Traceback (most recent call last):
web_1      |   File "/usr/lib/python3/dist-packages/odoo/tools/cache.py", line 85, in lookup
web_1      |     r = d[key]
web_1      |   File "<decorator-gen-6>", line 2, in __getitem__
web_1      |   File "/usr/lib/python3/dist-packages/odoo/tools/func.py", line 87, in locked
web_1      |     return func(inst, *args, **kwargs)
web_1      |   File "/usr/lib/python3/dist-packages/odoo/tools/lru.py", line 34, in __getitem__
web_1      |     a = self.d[obj]
web_1      | KeyError: ('ir.model.data', <function IrModelData._xmlid_lookup at 0x7ffb9ade4280>, 'base.module_payment_aps')
web_1      | 
web_1      | During handling of the above exception, another exception occurred:
web_1      | 
web_1      | Traceback (most recent call last):
web_1      |   File "/usr/lib/python3/dist-packages/odoo/tools/convert.py", line 698, in _tag_root
web_1      |     f(rec)
web_1      |   File "/usr/lib/python3/dist-packages/odoo/tools/convert.py", line 568, in _tag_record
web_1      |     f_val = self.id_get(f_ref, raise_if_not_found=nodeattr2bool(rec, 'forcecreate', True))
web_1      |   File "/usr/lib/python3/dist-packages/odoo/tools/convert.py", line 681, in id_get
web_1      |     res = self.model_id_get(id_str, raise_if_not_found)
web_1      |   File "/usr/lib/python3/dist-packages/odoo/tools/convert.py", line 687, in model_id_get
web_1      |     return self.env['ir.model.data']._xmlid_to_res_model_res_id(id_str, raise_if_not_found=raise_if_not_found)
web_1      |   File "/usr/lib/python3/dist-packages/odoo/addons/base/models/ir_model.py", line 2010, in _xmlid_to_res_model_res_id
web_1      |     return self._xmlid_lookup(xmlid)[1:3]
web_1      |   File "<decorator-gen-40>", line 2, in _xmlid_lookup
web_1      |   File "/usr/lib/python3/dist-packages/odoo/tools/cache.py", line 90, in lookup
web_1      |     value = d[key] = self.method(*args, **kwargs)
web_1      |   File "/usr/lib/python3/dist-packages/odoo/addons/base/models/ir_model.py", line 2003, in _xmlid_lookup
web_1      |     raise ValueError('External ID not found in the system: %s' % xmlid)
web_1      | ValueError: External ID not found in the system: base.module_payment_aps
web_1      | 
web_1      | The above exception was the direct cause of the following exception:
web_1      | 
web_1      | Traceback (most recent call last):
web_1      |   File "/usr/lib/python3/dist-packages/odoo/http.py", line 1987, in __call__
web_1      |     response = request._serve_db()
web_1      |   File "/usr/lib/python3/dist-packages/odoo/http.py", line 1562, in _serve_db
web_1      |     self.registry = Registry(self.db).check_signaling()
web_1      |   File "/usr/lib/python3/dist-packages/odoo/modules/registry.py", line 66, in __new__
web_1      |     return cls.new(db_name)
web_1      |   File "<decorator-gen-14>", line 2, in new
web_1      |   File "/usr/lib/python3/dist-packages/odoo/tools/func.py", line 87, in locked
web_1      |     return func(inst, *args, **kwargs)
web_1      |   File "/usr/lib/python3/dist-packages/odoo/modules/registry.py", line 90, in new
web_1      |     odoo.modules.load_modules(registry, force_demo, status, update_module)
web_1      |   File "/usr/lib/python3/dist-packages/odoo/modules/loading.py", line 483, in load_modules
web_1      |     processed_modules += load_marked_modules(cr, graph,
web_1      |   File "/usr/lib/python3/dist-packages/odoo/modules/loading.py", line 371, in load_marked_modules
web_1      |     loaded, processed = load_module_graph(
web_1      |   File "/usr/lib/python3/dist-packages/odoo/modules/loading.py", line 230, in load_module_graph
web_1      |     load_data(cr, idref, mode, kind='data', package=package)
web_1      |   File "/usr/lib/python3/dist-packages/odoo/modules/loading.py", line 71, in load_data
web_1      |     tools.convert_file(cr, package.name, filename, idref, mode, noupdate, kind)
web_1      |   File "/usr/lib/python3/dist-packages/odoo/tools/convert.py", line 763, in convert_file
web_1      |     convert_xml_import(cr, module, fp, idref, mode, noupdate)
web_1      |   File "/usr/lib/python3/dist-packages/odoo/tools/convert.py", line 829, in convert_xml_import
web_1      |     obj.parse(doc.getroot())
web_1      |   File "/usr/lib/python3/dist-packages/odoo/tools/convert.py", line 749, in parse
web_1      |     self._tag_root(de)
web_1      |   File "/usr/lib/python3/dist-packages/odoo/tools/convert.py", line 711, in _tag_root
web_1      |     raise ParseError('while parsing %s:%s, somewhere inside\n%s' % (
web_1      | odoo.tools.convert.ParseError: while parsing /usr/lib/python3/dist-packages/odoo/addons/payment/data/payment_provider_data.xml:23, somewhere inside
web_1      | <record id="payment_provider_aps" model="payment.provider">
web_1      |         <field name="name">Amazon Payment Services</field>
web_1      |         <field name="display_as">Amazon Payment Services</field>
web_1      |         <field name="image_128" type="base64" file="payment_aps/static/description/icon.png"/>
web_1      |         <field name="module_id" ref="base.module_payment_aps"/>
web_1      |         <!-- https://paymentservices.amazon.com/docs/EN/24.html -->
web_1      |         <field name="payment_icon_ids" eval="[(6, 0, [                         ref('payment.payment_icon_cc_mastercard'),                         ref('payment.payment_icon_cc_visa'),                         ref('payment.payment_icon_sadad'),                         ref('payment.payment_icon_mada'),                     ])]"/>
web_1      |     </record>

Seams i'm missing the payment_icon for Sadad and Mada. Those should be created by payment/data/payment_icon_data.xml but looking directly in the database the records are missing.

legalsylvain commented 1 year ago

hi, @ikus060, I don't get your point. If I understand correctly, you try to migrate a DB from 15.0 to 16.0 with 72 modules including payment module. Did you check the documentation, and the according coverage ? https://oca.github.io/OpenUpgrade/status.html It can not work at this moment.

ikus060 commented 1 year ago

Ho, I'm sorry. I did not get that from the empty table. That wasn't clear for me.

image

What is the plan to support the payment modules ? I think it's part of the basic module when you install the invoicing or accounting system.

jcdrubay commented 1 year ago

@ikus060 You can check the ongoing PR to see if anyone has already done some progress on these modules. If so, a code review would be highly welcomed. If no PR exists, then it's for you to make a plan. When do you plan to submit a PR ;)

ikus060 commented 1 year ago

@jcdrubay Having grasped the migration table, I comprehend that the migration from version 15 to 16 is still in its early stages. Considering the limited time available for me to carry out the Odoo migration, I cannot allocate all the necessary time to write the migration script. Moreover, I'm uncertain whether I possess the technical expertise required for scripting the migration process. Therefore, I will wait for some time until the migration is adequately covered.

barart commented 1 year ago

@ikus060 Hello, where do you find the steps to update the container? Im trying to do the same process as you (v15 to v16) running v15 on docker container but there is no mention of container in documentation.. so... how do you do the migration?

ikus060 commented 1 year ago

Upgrade postgresql version

When upgrading Odoo, you may need to also upgrade your postresql database to the latest version. 

docker-compose exec {service_name} pg_dumpall -U {postgres_user} > dump.sql

That will create a backup of your database as SQL file. You will need that file to restore your data into the newer postgresql.

Create a new volume or folder for the new postgresql database and edit your compose file to make use of this new volume or folder.

e.g.:

image: postgres:13
...
volumes:
  - ./db:/var/lib/postgresql/data

Should become

image: postgres:15
...
volumes:
  - ./db2:/var/lib/postgresql/data

NOTICE: make sure to add POSTGRES_HOST_AUTH_METHOD=md5 to you docker compose if you are migrating from postgresql:13 to postgresql:15 for backward compatibility.

Restart your docker compose.

docker-compose restart

Execute the following commands to restore your database into the new postgresql instance

docker-compose start
docker-compose exec {service_name} bash
psql -U {postgres_user} -d {default_postgres_database} < {mapped_volume_folder_path}/dump.sql

Finnaly, restart your docker compose again, and Odoo should be running with the new database.

Download OpenUpgrade to Odoo's Addons folder

cd ./data/addons
wget https://github.com/OCA/OpenUpgrade/archive/refs/heads/16.0.tar.gz
tar --strip-components=1 -zxvf 16.0.tar.gz OpenUpgrade-16.0/openupgrade_scripts OpenUpgrade-16.0/openupgrade_framework

Make sure the adjust the permissions of the folder when using Docker

chown -R 101:101 .

Update Odoo version and change command line in docker compose

Before restarting Odoo, update your docker compose file to the targeted version.

e.g.:

image: odoo:15.0

Should become 

image: odoo:16.0
command: bash -x -c "pip install https://github.com/OCA/openupgradelib/archive/refs/heads/master.zip && odoo --update all --load=base,web,openupgrade_framework"

That will have the efect to install openupgradelib dependencies within the image and to load openupgrade_framework addons as a server wide module.

Restart odoo

When using docker-compose

docker-compose restart

If you are using plain docker command line:

docker restart [your container name]

Initiate upgrade

Initiate the upgrade by browsing to odoo url.

http://localhost:8069

The log should display multiple SQL statements to upgrade your existing database.

Need to create column

UPDATE public.mail_mail SET scheduled_date = NULL WHERE scheduled_date = '';

db_1 | 2023-05-03 16:56:13.169 UTC [42] ERROR: could not create unique index "utm_source_unique_name" db_1 | 2023-05-03 16:56:13.169 UTC [42] DETAIL: Key (name)=(rdiffweb.org : where to find everything about the simplified backup management software 2022-06-13 12:40:55 (copie) (copie) (copie)) is duplicated. db_1 | 2023-05-03 16:56:13.169 UTC [42] STATEMENT: ALTER TABLE "utm_source" ADD CONSTRAINT "utm_source_unique_name" UNIQUE(name) db_1 | 2023-05-03 16:56:13.907 UTC [42] ERROR: invalid input syntax for type timestamp: "" db_1 | 2023-05-03 16:56:13.907 UTC [42] STATEMENT:
db_1 | ALTER TABLE "mail_mail" db_1 | ALTER COLUMN "scheduled_date" DROP DEFAULT, db_1 | ALTER COLUMN "scheduled_date" TYPE timestamp USING "scheduled_date"::timestamp

barart commented 1 year ago

@ikus060 THanks for your reply, i just needed to modify something on your steps:

i needed to delete this two sequences on the database:

base_cache_signaling base_registry_signaling

And then change the command:

the command should be this: bash -x -c "apt update && apt install -y git && pip install -U pip && pip install git+https://github.com/OCA/openupgradelib.git@master#egg=openupgradelib && odoo --update all --load=base,web,openupgrade_framework"

Now, im stuck on the payment module too, i see that there is a PR for it: https://github.com/OCA/OpenUpgrade/pull/3918 but its not approved,

joergstrohmeier commented 11 months ago

I am so happy that I found this post for upgrading my docker instances. Now I wanted to test the commands in a docker shell but no permissin or not installed. Any suggestions?

$ apt update Reading package lists... Done E: List directory /var/lib/apt/lists/partial is missing. - Acquire (13: Permission denied) odoo@odoo_prod_v14:/$ apt install -y git E: Could not open lock file /var/lib/dpkg/lock-frontend - open (13: Permission denied) E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), are you root? odoo@odoo_prod_v14:/$ pip install bash: pip: command not found

joergstrohmeier commented 11 months ago

and yes, obviously the upgrade failed because the module could not be loaded ERROR ? odoo.service.server: Failed to load server-wide module openupgrade_framework. CRITICAL ? odoo.modules.module: No module named 'openupgradelib' CRITICAL ? odoo.modules.module: Couldn't load module openupgrade_framework + odoo --update all --stop-after-init --load=base,web,openupgrade_framework

when I start docker shell I am connected as user "odoo"

ikus060 commented 11 months ago

I'm still in progress to migrate my Odoo instance. I'm waiting for a couple of module to get updated.

But here the files I used to upgrade.

odoo-upgrade.zip

upgrade.sh is the script launched to automate the upgrade. Read comments to understand each step. docker-compose-old.yaml is my current docker compose deployement docker-compose.yaml is the new docker compose deployment (odoo16 + postgresql update)

joergstrohmeier commented 11 months ago

In the meantime I found a workaround for my root issue. Have to mention that I am on a Synology NAS with Container Manager module, so I dont wanted to do much on the Linux command line. I have used potainer to manage docker and with that I could connect with a docker shell as root and run the installations of git and opernupgradelib < super that you posted this command her!! I had to change to from pip to pip3 and the pip3 update did not run. With the remaining odoo udate all in the command section of the docker compose file it startet. My upgrade from 14 to 15 worked. I had to manipulate the payments service provider data file as there seemed to be wrong record in there. But at the end the log contains still some errors and warnings so I will still have to consult a specialist who has experience in this and can interpret them. I tried 15 to 16 but that got stuck with payments service provider file again (!) and other severe error that brought the upgrade to stop. In your docker compose that you posted in here are you sure that the deploy order: start-first is correct? In your final files you do not use this section any more? And you removed the db, user and password paramters, to the odoo.config file I assume. I also have it in there - seems to work better than in the container for what ever reason.