search

nix-community / dream2nix

Simplified nix packaging for various programming language ecosystems [maintainer=@DavHau]
https://dream2nix.dev
MIT License
975 stars 122 forks source link

v1/nix/modules/drvs/odoo # ImportError: cannot import name 'utils' from 'PyPDF2' (/nix/store/d-python3.8-pypdf2-3.0.1/lib/python3.8/site-packages/PyPDF2/__init__.py) #537

Open InLaw opened 1 year ago

InLaw commented 1 year ago

is there a reason why python is in some packages fixed to python38 and in others to python39?

/nix/store/xl6q8j50a5lcf0ixxpxnlyvnxk7ybd3l-python3.8-odoo-16.0/bin/odoo

/nix/store/xl6q8j50a5lcf0ixxpxnlyvnxk7ybd3l-python3.8-odoo-16.0/bin/odoo
2023-06-28 13:39:23,536 448402 INFO ? odoo: Odoo version 16.0 
2023-06-28 13:39:23,536 448402 INFO ? odoo: addons paths: ['/nix/store/xl6q8j50a5lcf0ixxpxnlyvnxk7ybd3l-python3.8-odoo-16.0/lib/python3.8/site-packages/odoo/addons', '/home/ae/.local/share/Odoo/addons/16.0'] 
2023-06-28 13:39:23,536 448402 INFO ? odoo: database: default@default:default 
2023-06-28 13:39:23,537 448402 CRITICAL ? odoo.modules.module: Couldn't load module web 
2023-06-28 13:39:23,537 448402 CRITICAL ? odoo.modules.module: No module named 'odoo.addons.web' 
2023-06-28 13:39:23,537 448402 ERROR ? odoo.service.server: Failed to load server-wide module `web`.
The `web` module is provided by the addons found in the `openerp-web` project.
Maybe you forgot to add those addons in your addons_path configuration. 
Traceback (most recent call last):
  File "/nix/store/xl6q8j50a5lcf0ixxpxnlyvnxk7ybd3l-python3.8-odoo-16.0/lib/python3.8/site-packages/odoo/service/server.py", line 1247, in load_server_wide_modules
    odoo.modules.module.load_openerp_module(m)
  File "/nix/store/xl6q8j50a5lcf0ixxpxnlyvnxk7ybd3l-python3.8-odoo-16.0/lib/python3.8/site-packages/odoo/modules/module.py", line 432, in load_openerp_module
    __import__('odoo.addons.' + module_name)
ModuleNotFoundError: No module named 'odoo.addons.web'
2023-06-28 13:39:23,604 448402 CRITICAL ? odoo.modules.module: Couldn't load module base 
2023-06-28 13:39:23,604 448402 CRITICAL ? odoo.modules.module: cannot import name 'utils' from 'PyPDF2' (/nix/store/dyn4l9sb94d8139h18z7dmdm52681s9l-python3.8-pypdf2-3.0.1/lib/python3.8/site-packages/PyPDF2/__init__.py) 
2023-06-28 13:39:23,604 448402 ERROR ? odoo.service.server: Failed to load server-wide module `base`. 
Traceback (most recent call last):
  File "/nix/store/xl6q8j50a5lcf0ixxpxnlyvnxk7ybd3l-python3.8-odoo-16.0/lib/python3.8/site-packages/odoo/service/server.py", line 1247, in load_server_wide_modules
    odoo.modules.module.load_openerp_module(m)
  File "/nix/store/xl6q8j50a5lcf0ixxpxnlyvnxk7ybd3l-python3.8-odoo-16.0/lib/python3.8/site-packages/odoo/modules/module.py", line 432, in load_openerp_module
    __import__('odoo.addons.' + module_name)
  File "/nix/store/xl6q8j50a5lcf0ixxpxnlyvnxk7ybd3l-python3.8-odoo-16.0/lib/python3.8/site-packages/odoo/addons/base/__init__.py", line 5, in <module>
    from . import models
  File "/nix/store/xl6q8j50a5lcf0ixxpxnlyvnxk7ybd3l-python3.8-odoo-16.0/lib/python3.8/site-packages/odoo/addons/base/models/__init__.py", line 11, in <module>
    from . import ir_actions_report
  File "/nix/store/xl6q8j50a5lcf0ixxpxnlyvnxk7ybd3l-python3.8-odoo-16.0/lib/python3.8/site-packages/odoo/addons/base/models/ir_actions_report.py", line 25, in <module>
    from PyPDF2 import PdfFileWriter, PdfFileReader, utils
ImportError: cannot import name 'utils' from 'PyPDF2' (/nix/store/dyn4l9sb94d8139h18z7dmdm52681s9l-python3.8-pypdf2-3.0.1/lib/python3.8/site-packages/PyPDF2/__init__.py)
2023-06-28 13:39:23,608 448402 INFO ? odoo.service.server: HTTP service (werkzeug) running on localhost:8069 
Exception in thread odoo.service.cron.cron0:
Traceback (most recent call last):
  File "/nix/store/p360ckl7a6j3svz50v0bbligi9i45hjp-python3-3.8.17/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/nix/store/p360ckl7a6j3svz50v0bbligi9i45hjp-python3-3.8.17/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/nix/store/xl6q8j50a5lcf0ixxpxnlyvnxk7ybd3l-python3.8-odoo-16.0/lib/python3.8/site-packages/odoo/service/server.py", line 496, in target
    self.cron_thread(i)
  File "/nix/store/xl6q8j50a5lcf0ixxpxnlyvnxk7ybd3l-python3.8-odoo-16.0/lib/python3.8/site-packages/odoo/service/server.py", line 452, in cron_thread
    from odoo.addons.base.models.ir_cron import ir_cron
  File "/nix/store/xl6q8j50a5lcf0ixxpxnlyvnxk7ybd3l-python3.8-odoo-16.0/lib/python3.8/site-packages/odoo/addons/base/__init__.py", line 5, in <module>
    from . import models
  File "/nix/store/xl6q8j50a5lcf0ixxpxnlyvnxk7ybd3l-python3.8-odoo-16.0/lib/python3.8/site-packages/odoo/addons/base/models/__init__.py", line 11, in <module>
    from . import ir_actions_report
  File "/nix/store/xl6q8j50a5lcf0ixxpxnlyvnxk7ybd3l-python3.8-odoo-16.0/lib/python3.8/site-packages/odoo/addons/base/models/ir_actions_report.py", line 25, in <module>
    from PyPDF2 import PdfFileWriter, PdfFileReader, utils
ImportError: cannot import name 'utils' from 'PyPDF2' (/nix/store/dyn4l9sb94d8139h18z7dmdm52681s9l-python3.8-pypdf2-3.0.1/lib/python3.8/site-packages/PyPDF2/__init__.py)
Exception in thread odoo.service.cron.cron1:
Traceback (most recent call last):
  File "/nix/store/p360ckl7a6j3svz50v0bbligi9i45hjp-python3-3.8.17/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/nix/store/p360ckl7a6j3svz50v0bbligi9i45hjp-python3-3.8.17/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/nix/store/xl6q8j50a5lcf0ixxpxnlyvnxk7ybd3l-python3.8-odoo-16.0/lib/python3.8/site-packages/odoo/service/server.py", line 496, in target
    self.cron_thread(i)
  File "/nix/store/xl6q8j50a5lcf0ixxpxnlyvnxk7ybd3l-python3.8-odoo-16.0/lib/python3.8/site-packages/odoo/service/server.py", line 452, in cron_thread
    from odoo.addons.base.models.ir_cron import ir_cron
  File "/nix/store/xl6q8j50a5lcf0ixxpxnlyvnxk7ybd3l-python3.8-odoo-16.0/lib/python3.8/site-packages/odoo/addons/base/__init__.py", line 5, in <module>
    from . import models
  File "/nix/store/xl6q8j50a5lcf0ixxpxnlyvnxk7ybd3l-python3.8-odoo-16.0/lib/python3.8/site-packages/odoo/addons/base/models/__init__.py", line 11, in <module>
    from . import ir_actions_report
  File "/nix/store/xl6q8j50a5lcf0ixxpxnlyvnxk7ybd3l-python3.8-odoo-16.0/lib/python3.8/site-packages/odoo/addons/base/models/ir_actions_report.py", line 25, in <module>
    from PyPDF2 import PdfFileWriter, PdfFileReader, utils
ImportError: cannot import name 'utils' from 'PyPDF2' (/nix/store/dyn4l9sb94d8139h18z7dmdm52681s9l-python3.8-pypdf2-3.0.1/lib/python3.8/site-packages/PyPDF2/__init__.py)
phaer commented 1 year ago

I can reproduce this. Exception means that the PyPDF used is too new for this build of odoo. Haven't looked into fixing this yet; Not sure when it changed either.

InLaw commented 1 year ago

https://github.com/odoo/odoo/blob/16.0/requirements.txt#L42C1-L43C41

PyPDF2==1.26.0 ; python_version <= '3.10'
PyPDF2==2.12.1 ; python_version > '3.10'

/odoo/lock-x86_64-linux.json

 "pypdf2": {
        "sha256": "d16e4205cfee272fbdc0568b68d82be796540b1537508cef59388f839c191928",
        "url": "https://files.pythonhosted.org/packages/8e/5e/c86a5643653825d3c913719e788e41386bee415c2b87b4f955432f2de6b2/pypdf2-3.0.1-py3-none-any.whl",
        "version": "3.0.1"
      },

dream2nix ignores the requirements/specifications (currently), right?

InLaw commented 1 year ago

how would you patch pypdf2 in pip = { drvs = { ? As workaround and easier/ad-hoc would be postPatch it

phaer commented 1 year ago

dream2nix ignores the requirements/specifications (currently), right? it purely takes all packages (by name) from pypiSnapshotDate = "2023-04-01

No, it respects version, environment and extra markers. There's most probably something broken w.r.t. to the resolution at the moment, but we'll need to look into it a bit closer.

how would you patch pypdf2 in pip = { drvs = { ?

It uses drv parts. See odoos default.nix for an example of attribute overrides, i.e. doCheck for pypdf2 is currently set to false there. You should be able to use `pip.drvs.pypdf2.mkDerivation.postPatch

InLaw commented 1 year ago

when trying pip.drvs.pypdf2.mkDerivation.postPatch

the node is not yet created in nix/store mv: cannot stat '/nix/store/k7fwq6fmpwzasasyxy3mw5vwr1gpj6j5-python3.11-pypdf2-3.0.1/lib/python3.11/site-packages/PyPDF2/_utils.py': No such file or directory

Where would be the place (in the dream2nix logic/workflow) for the package file renaming if pip.drvs.pypdf2.mkDerivation "only" creates a whl file?

phaer commented 1 year ago

Still a problem on latest main.

dream2nix ignores the requirements/specifications (currently), right?

To elaborate on that: It respects those markers, but we currently only rely on the markers in odoos setup.py/setup.cfg/pyproject, we don't use requirements.txt

And https://github.com/odoo/odoo/blob/2d42fd69cada3b1f2716c3d0a20bec6170f9b226/setup.py mentions only "PyPDF2" thats why we are getting the most recent version here.

Will try to free some time to look into the details of switching to requirements.txt for this use-case, as I'll review https://github.com/nix-community/dream2nix/pull/543/files which deals with a similar issue.