search

acsone / setuptools-odoo

A library to help packaging Odoo addons with setuptools. It mainly populates the usual setup.py keywords from the Odoo manifest files.
GNU Lesser General Public License v3.0
41 stars 45 forks source link

FileNotFoundError addons-7c.txt #55

Open simahawk opened 3 years ago

simahawk commented 3 years ago

Versions

Python: 3.8 setuptools-odoo: 2.7.1 (installed w/ pip3 install --user setuptools-odoo) pre-commit: 2.13.0 (installed w/ pip3 install --user pre-commit) Odoo version: 14.0

I get this when running pre-commit:

FileNotFoundError: [Errno 2] No such file or directory: '/home/sorsi/.cache/pre-commit/repo1mxdepkf/py_env-python3/lib/python3.8/site-packages/setuptools_odoo/addons-7c.txt'

I find the file in /home/sorsi/.cache/pre-commit/repo1mxdepkf/setuptools_odoo/addons-7c.txt so it seems that the lookup for these txt files is broken here https://github.com/acsone/setuptools-odoo/blob/f481c16191b92c1dc026ea950a38768198a3a663/setuptools_odoo/base_addons.py#L11

Maybe it happens only w/ pre-commit + 3.8?

sbidoul commented 3 years ago

That's weird. Have you tried clearing your pre-commit cache ?

simahawk commented 3 years ago

it's a brand new setup from scratch but I'll try!

simahawk commented 3 years ago

@sbidoul tried again: no luck. I had to copy files there to make it work.

I'm in a virtualenv and I thought this was the issue but I've tried inside and outside it, no luck.

I've tried uninstalling venv installation of setuptools-odoo and pre-commit, no luck.

I've tried cleaning the cache in both cases, no luck.

# which pre-commit
/home/sorsi/.local/bin/pre-commit
# which setuptools-odoo-make-default 
/home/sorsi/.local/bin/setuptools-odoo-make-default
sbidoul commented 3 years ago

Can you share the relevant pre-commit config? I'll give it a try.

simahawk commented 3 years ago

https://github.com/OCA/product-attribute/blob/14.0/.pre-commit-config.yaml

same result w/ https://github.com/shopinvader/odoo-shopinvader/blob/14.0/.pre-commit-config.yaml

sbidoul commented 3 years ago

Note that which setuptools-odoo-make-default should be irrelevant, since pre-commit installs it and runs it in an isolated virtualenv.

sbidoul commented 3 years ago

Perhaps the fact that you installed both pre-commit and setuptools-odoo with pip install --user is confusing precommit ? FWIW, the only thing I personally install with --user is pipx, and then I pipx install pre-commit and all other tools from PyPI.

sebalix commented 3 years ago

I have the same issue, also on a fresh install/pre-commit cache, it says:

Generate default setup.py for an addons directory........................Failed
- hook id: setuptools-odoo-make-default
- exit code: 1

Traceback (most recent call last):
  File "/home/salix/.cache/pre-commit/reporh098zcx/py_env-python3/bin/setuptools-odoo-make-default", line 5, in <module>
    from setuptools_odoo.make_default_setup import main
  File "/home/salix/.cache/pre-commit/reporh098zcx/py_env-python3/lib/python3.8/site-packages/setuptools_odoo/__init__.py", line 6, in <module>
    from .core import get_addon_metadata
  File "/home/salix/.cache/pre-commit/reporh098zcx/py_env-python3/lib/python3.8/site-packages/setuptools_odoo/core.py", line 14, in <module>
    from . import base_addons, external_dependencies
  File "/home/salix/.cache/pre-commit/reporh098zcx/py_env-python3/lib/python3.8/site-packages/setuptools_odoo/base_addons.py", line 16, in <module>
    openerp7 = _addons("7c")
  File "/home/salix/.cache/pre-commit/reporh098zcx/py_env-python3/lib/python3.8/site-packages/setuptools_odoo/base_addons.py", line 12, in _addons
    b = resource_string("setuptools_odoo", "addons-%s.txt" % suffix)
  File "/home/salix/.cache/pre-commit/reporh098zcx/py_env-python3/lib/python3.8/site-packages/pkg_resources/__init__.py", line 1142, in resource_string
    return get_provider(package_or_requirement).get_resource_string(
  File "/home/salix/.cache/pre-commit/reporh098zcx/py_env-python3/lib/python3.8/site-packages/pkg_resources/__init__.py", line 1388, in get_resource_string
    return self._get(self._fn(self.module_path, resource_name))
  File "/home/salix/.cache/pre-commit/reporh098zcx/py_env-python3/lib/python3.8/site-packages/pkg_resources/__init__.py", line 1611, in _get
    with open(path, 'rb') as stream:
FileNotFoundError: [Errno 2] No such file or directory: '/home/salix/.cache/pre-commit/reporh098zcx/py_env-python3/lib/python3.8/site-packages/setuptools_odoo/addons-7c.txt'

pre-commit config: https://github.com/OCA/wms/blob/3d7446f47af2ec134ae14c6bc3d264b406d242cc/.pre-commit-config.yaml And indeed pre-commit is installed with --user (but not setuptools-odoo, which is only installed in the pre-commit hook Python environment).

sbidoul commented 3 years ago

Can you try to create a reproducer in a container or something ?

sbidoul commented 2 years ago

Is this still an issue ?

simahawk commented 2 years ago

Not AFAIK

StefanRijnhart commented 2 years ago

For the record, I' running into this. After clearing the cache and reinitializing pre-commit, I have the following versions:

(py3.8) $ . /home/odoo/.cache/pre-commit/repo4_14zsr6/py_env-python3/bin/activate
(py_env-python3) bruno@tulpa:~/emesa/oca/web$ pip list
Package         Version
--------------- -----------------
packaging       21.3
pip             22.1.2
pyparsing       3.0.9
setuptools      62.2.0
setuptools-odoo 0.1.dev1+gd744fc6
setuptools-scm  6.4.2
tomli           2.0.1
wheel           0.37.1

I can activate pre-commit's virtualenv and upgrade setuptools-odoo which fixes the problem, but it's not ideal.

sbidoul commented 2 years ago

@StefanRijnhart this looks like setuptools-odoo was installed from git or a git checkout ? That's weird. I can't really do much without a reproducer.

simahawk commented 2 years ago

@sbidoul I'm hitting this again on a setup from scratch.

Traceback (most recent call last):
  File "/home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/bin/setuptools-odoo-make-default", line 5, in <module>
    from setuptools_odoo.make_default_setup import main
  File "/home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/lib/python3.10/site-packages/setuptools_odoo/__init__.py", line 6, in <module>
    from .core import get_addon_metadata
  File "/home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/lib/python3.10/site-packages/setuptools_odoo/core.py", line 14, in <module>
    from . import base_addons, external_dependencies
  File "/home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/lib/python3.10/site-packages/setuptools_odoo/base_addons.py", line 16, in <module>
    openerp7 = _addons("7c")
  File "/home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/lib/python3.10/site-packages/setuptools_odoo/base_addons.py", line 12, in _addons
    b = resource_string("setuptools_odoo", "addons-%s.txt" % suffix)
  File "/home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/lib/python3.10/site-packages/pkg_resources/__init__.py", line 1166, in resource_string
    return get_provider(package_or_requirement).get_resource_string(
  File "/home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/lib/python3.10/site-packages/pkg_resources/__init__.py", line 1412, in get_resource_string
    return self._get(self._fn(self.module_path, resource_name))
  File "/home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/lib/python3.10/site-packages/pkg_resources/__init__.py", line 1635, in _get
    with open(path, 'rb') as stream:
FileNotFoundError: [Errno 2] No such file or directory: '/home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/lib/python3.10/site-packages/setuptools_odoo/addons-7c.txt'

pre-commit 2.20.0


$ /home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/bin/pip list
Package           Version
----------------- -----------------
packaging         21.3
pip               22.2.2
pyparsing         3.0.9
setuptools        65.1.1
setuptools-odoo   0.1.dev1+gd744fc6
setuptools-scm    7.0.5
tomli             2.0.1
typing_extensions 4.3.0
wheel             0.37.1
``
simahawk commented 2 years ago

I confirm that upgrading setuptools-odoo fixes the problem

$ /home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/bin/pip install setuptools-odoo
Requirement already satisfied: setuptools-odoo in /home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/lib/python3.10/site-packages (0.1.dev1+gd744fc6)
Requirement already satisfied: setuptools-scm!=4.0.0,>=2.1 in /home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/lib/python3.10/site-packages (from setuptools-odoo) (7.0.5)
Requirement already satisfied: setuptools in /home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/lib/python3.10/site-packages (from setuptools-odoo) (65.1.1)
Requirement already satisfied: packaging>=20.0 in /home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/lib/python3.10/site-packages (from setuptools-scm!=4.0.0,>=2.1->setuptools-odoo) (21.3)
Requirement already satisfied: tomli>=1.0.0 in /home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/lib/python3.10/site-packages (from setuptools-scm!=4.0.0,>=2.1->setuptools-odoo) (2.0.1)
Requirement already satisfied: typing-extensions in /home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/lib/python3.10/site-packages (from setuptools-scm!=4.0.0,>=2.1->setuptools-odoo) (4.3.0)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/lib/python3.10/site-packages (from packaging>=20.0->setuptools-scm!=4.0.0,>=2.1->setuptools-odoo) (3.0.9)
(cos14) $ sorsi@pop-os (14-edi-drop-ack-create *+) ~/dev/projects/cosanum_odoo/odoo/external-src/edi 
$ /home/sorsi/.cache/pre-commit/repo_lofcqd5/py_env-python3/bin/pip install setuptools-odoo --upgrade
[...]
Installing collected packages: setuptools-odoo
  Attempting uninstall: setuptools-odoo
    Found existing installation: setuptools-odoo 0.1.dev1+gd744fc6
    Uninstalling setuptools-odoo-0.1.dev1+gd744fc6:
      Successfully uninstalled setuptools-odoo-0.1.dev1+gd744fc6
Successfully installed setuptools-odoo-3.1.4

So for some reasons, pre-commit installs 0.1.dev1+gd744fc6 instead of the latest version :thinking:

pre-commit config https://github.com/OCA/edi/blob/14.0/.pre-commit-config.yaml

sbidoul commented 2 years ago

This setuptools-odoo 0.1.dev1+gd744fc6 is not on PyPI, so you'll have to find out how it ended up in your pre-commit env.

simahawk commented 2 years ago

I know but I have no clue :sweat_smile: I'm working on a brand new laptop where I had to re-setup from scratch my projects' envs. Here I just ran pre-commit install and committed something. Also, here we have no control on how setuptools-odoo is installed because is done by pre-commit :man_shrugging:

sbidoul commented 2 years ago

Commit d744fc6 (of 0.1.dev1+gd744fc6) corresponds to the 2.6.0 tag.

sbidoul commented 2 years ago

Could you try manually reinstalling setuptools-odoo==2.6.0 in the pre-commit generated venv?

sbidoul commented 2 years ago

The fact that it shows version 0.1.dev1+gd744fc6 could be because pre-commit does not fetch tags, but that should not prevent setuptools to work normally.

sbidoul commented 2 years ago

If a repro situation, can you also list the content of {thevenv}/python3.10/site-packages/setuptools_odoo ?

max3903 commented 1 year ago

Same problem here.

sbidoul commented 1 year ago

Installing pre-commit with pipx and emptying the pre-commit cache fixed the issue.

It is likely due to some incompatibility with the system setuptools.

dreispt commented 1 year ago

Getting the same problem, after running copier update on an OCA repo. As a workaround, I'm setting the setuptools-odoo version back to 2.6.0.

sbidoul commented 1 year ago

The best explanation we have is an incompatibility somewhere between pre-commit, setuptools_scm and the system python packages.

The recommendation is to install pre-commit using pipx.

dreispt commented 1 year ago

I actually already had pre-commit with pipx. I ran:

pip uninstall pre-commit
pipx uninstall pre-commit
pipx install pre-commit

Still get same error.

Not sure if this helps:

$ pipx list
venvs are in /home/daniel/.local/pipx/venvs
apps are exposed on your $PATH at /home/daniel/.local/bin
   package copier 5.1.0, installed using Python 3.10.4
    - copier (symlink missing or pointing to unexpected location)
   package pre-commit 2.20.0, installed using Python 3.10.6
    - pre-commit
    - pre-commit-validate-config
    - pre-commit-validate-manifest
dreispt commented 1 year ago

Addition info - from trial and error, it stops working for me at setuptools-odoo 3.1.5. Last version that works is 3.1.4.

sbidoul commented 1 year ago

Addition info - from trial and error, it stops working for me at setuptools-odoo 3.1.5. Last version that works is 3.1.4.

That is weird. The diff between these two version has really nothing that could explain that.

dreispt commented 1 year ago

Some imports changed. Maybe that activated code that triggers the issue?

dreispt commented 1 year ago

I dug into the virtualenv being used: activated it, ran setuptools-odoo-make-default, and saw the error happening. The error happens at pkg_resources/__init__.py +1636, running with open(path, 'rb') as stream: The value in path is .../site-packages/setuptools_odoo/addons-7c.txt. But no txt files exist there - the directory content is:

base_addons.py  external_dependencies.py  git_postversion.py  make_default_setup.py  __pycache__
core.py         get_requirements.py       __init__.py         manifest.py            setup_keywords.py

Can this be a packaging issue?

dreispt commented 1 year ago

In case it helps, venv package list:

$ pip list
Package           Version
----------------- -----------------
packaging         21.3
pip               22.3.1
pyparsing         3.0.9
setuptools        65.5.1
setuptools-odoo   0.1.dev1+gecdc64b
setuptools-scm    7.0.5
tomli             2.0.1
typing_extensions 4.4.0
wheel             0.37.1
sbidoul commented 1 year ago

Some imports changed.

I'm confused: this is the diff I see between 3.1.4 and 3.1.5

image

sbidoul commented 1 year ago

Again, I'd be happy to investigate but I can't reproduce. If someone can reproduce in a Dockerfile or something I'll be happy to dig into it.

dreispt commented 1 year ago

Found a solution that worked for me - turn off your computer and turn it on again ... or almost that 😸 : Just run pre-commit clean, and after that pre-commit run -a worked fine.

simahawk commented 1 year ago

Found a solution that worked for me - turn off your computer and turn it on again ... or almost that smile_cat : Just run pre-commit clean, and after that pre-commit run -a worked fine.

pre-commit clean does not fix anything for me :sob:

I just did this:

  1. uninstalled pre-commit from my pyenv / virtualenv
  2. installed it from scratch usin pipx
  3. ran pre-commit clean
  4. run again -> same error :man_shrugging:
simahawk commented 1 year ago 
FileNotFoundError: [Errno 2] No such file or directory: '/home/sorsi/.cache/pre-commit/repope1kgfmy/py_env-python3/lib/python3.10/site-packages/setuptools_odoo/addons-7c.txt'

$ find -name addons-7c.txt
./repope1kgfmy/setuptools_odoo/addons-7c.txt

The issue seems to be that somehow the file gets located in the wrong place...

simahawk commented 1 year ago

@sbidoul BTW I'm on v14, what's the point of loading all the versions? (I'm checking)

simahawk commented 1 year ago 
$ find -name setuptools_odoo
./pre-commit/repod6fj5j15/setuptools_odoo
./pre-commit/repod6fj5j15/build/lib/setuptools_odoo
./pre-commit/repod6fj5j15/py_env-python3/lib/python3.10/site-packages/setuptools_odoo
$ tree ./pre-commit/repod6fj5j15/setuptools_odoo
./pre-commit/repod6fj5j15/setuptools_odoo
├── addons-10c.txt
├── addons-10e.txt
├── addons-11c.txt
├── addons-11e.txt
├── addons-12c.txt
├── addons-12e.txt
├── addons-13c.txt
├── addons-13e.txt
├── addons-14c.txt
├── addons-14e.txt
├── addons-15c.txt
├── addons-15e.txt
├── addons-16c.txt
├── addons-16e.txt
├── addons-7c.txt
├── addons-8c.txt
├── addons-9c.txt
├── addons-9e.txt
├── base_addons.py
├── core.py
├── external_dependencies.py
├── get_requirements.py
├── git_postversion.py
├── __init__.py
├── make_default_setup.py
├── manifest.py
└── setup_keywords.py
$ tree ./pre-commit/repod6fj5j15/build/lib/setuptools_odoo/
./pre-commit/repod6fj5j15/build/lib/setuptools_odoo/
├── base_addons.py
├── core.py
├── external_dependencies.py
├── get_requirements.py
├── git_postversion.py
├── __init__.py
├── make_default_setup.py
├── manifest.py
└── setup_keywords.py

$ tree ./pre-commit/repod6fj5j15/py_env-python3/lib/python3.10/site-packages/setuptools_odoo
./pre-commit/repod6fj5j15/py_env-python3/lib/python3.10/site-packages/setuptools_odoo
├── base_addons.py
├── core.py
├── external_dependencies.py
├── get_requirements.py
├── git_postversion.py
├── __init__.py
├── make_default_setup.py
├── manifest.py
├── __pycache__
│   ├── base_addons.cpython-310.pyc
│   ├── core.cpython-310.pyc
│   ├── external_dependencies.cpython-310.pyc
│   ├── get_requirements.cpython-310.pyc
│   ├── git_postversion.cpython-310.pyc
│   ├── __init__.cpython-310.pyc
│   ├── make_default_setup.cpython-310.pyc
│   ├── manifest.cpython-310.pyc
│   └── setup_keywords.cpython-310.pyc
└── setup_keywords.py

This looks insane.

simahawk commented 1 year ago 
> /home/sorsi/.cache/pre-commit/repod6fj5j15/py_env-python3/lib/python3.10/site-packages/pkg_resources/__init__.py(351)get_provider()
-> def get_provider(moduleOrReq):
> /home/sorsi/.cache/pre-commit/repod6fj5j15/py_env-python3/lib/python3.10/site-packages/pkg_resources/__init__.py(1411)

1411  ->     def get_resource_string(self, manager, resource_name):                                                                                                                      
1412             return self._get(self._fn(self.module_path, resource_name))                                                                                                             
(Pdb++) self
<pkg_resources.DefaultProvider object at 0x7f33dd141630>
(Pdb++) self.module_path
'/home/sorsi/.cache/pre-commit/repod6fj5j15/py_env-python3/lib/python3.10/site-packages/setuptools_odoo'

As you see, inside the pkg_resource lookup, the module path is the latter, which is missing those files.

dreispt commented 1 year ago

@simahawk Sharing my result:

$ find -name addons-7c.txt
./repohr54hezv/build/lib/setuptools_odoo/addons-7c.txt
./repohr54hezv/setuptools_odoo/addons-7c.txt
./repohr54hezv/py_env-python3/lib/python3.10/site-packages/setuptools_odoo/addons-7c.txt
./repobiz5uwfh/build/lib/setuptools_odoo/addons-7c.txt
./repobiz5uwfh/setuptools_odoo/addons-7c.txt
./repobiz5uwfh/py_env-python3/lib/python3.10/site-packages/setuptools_odoo/addons-7c.txt
./repo469mamt8/setuptools_odoo/addons-7c.txt
./repof8w2ztgf/setuptools_odoo/addons-7c.txt

I have four caches, all created recently (because I recently ran pre-commit clean). In two of them I see the same result as you, in two others I see the file also in lib and py_env.

I have no idea if this is helpful...

simahawk commented 1 year ago

@sbidoul once again getting back to this... Here's a brand new setup cleaned w/ pre-commit clean:

setuptools-broken-txt-position

After my last cp command it's solved. See also https://github.com/acsone/setuptools-odoo/issues/55#issuecomment-1318413896

sbidoul commented 1 year ago

@simahawk could you do a Dockerfile with a reproducer?

simahawk commented 1 year ago

I'll try, not sure when :sweat_smile:

cuongnmtm commented 1 year ago

I am facing this issue and trying to debug.

There is no data file in the cache

image

So I try to check why data files are not copied when setuptools try to setup the package. I checked the code and it looks good with include_package_data=True in setuptools.setup().

Then I research on the internet and found this https://stackoverflow.com/questions/7522250/how-to-include-package-data-with-setuptools-distutils/14159430#14159430 I am not sure the answer is related to this issue or not, but gave it a try. I added include addons*.txt to MANIFEST.in and it seems it solved my issue

image
sbidoul commented 1 year ago

@cuongnmtm since setuptools-odoo uses the git file finder via setuptools_scm, updating MANIFEST.in should not be necessary.

By chance do you have a procedure to reproduce the issue?

cuongnmtm commented 1 year ago

@sbidoul I just tried to reproduce the issue but cannot...

I tried to run pre-commit clean and pre-commit run -a, no issue now. But at the time I faced the issue, this didn't solve the issue.

This only happens with very few of my projects (actually just one of two projects). Let's see if I faced this issue again in the future.

Thank you.

SirAionTech commented 5 months ago

I faced this exact issue about file "addons-7c.txt" and another one about the readme generation, that complained about a missing "gen_addon_readme.rst.jinja" somewhere.

Re-installing pre-commit fixed it for me:

pre-commit clean
pipx uninstall pre-commit
pipx install pre-commit

Now pre-commit run --all-files works perfectly.

Hope this helps :smile: