radiasoft / pykern

Apache License 2.0
5 stars 7 forks source link

Replace pkg_resources with importlib #462

Open robnagler opened 5 months ago

robnagler commented 5 months ago

This is really screwed up. They are adding things in 3.9 and deprecating in 3.12. We are going to have to change the model of pkresource to implement some form of caching of data, which aligns with how importlib.resources works. We'll have to provide backwards compatibility, because there's an assumption that the files can be in a wheel (zip) so you can't just symlink to those files.

I think at this time we can add a hook to pkcli to call clean up functions. I don't think pykern functions are invoked directly (e.g. uswgi) so they always start from a pkcli. We could check for this in side pkresource.

pkresource will create a temporary directory (in /tmp is probably best, especially for docker) that caches resources and removes them on program exit (with pkcli hook).

Here's something that's really broken:

$ cd ~/src/radiasoft
$ python
Python 3.9.15 (main, Jan  5 2024, 23:53:42)
[GCC 12.2.1 20221121 (Red Hat 12.2.1-4)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import importlib.resources as r
>>> for f in r.files("pykern").iterdir():
...     print(f)
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/vagrant/.pyenv/versions/3.9.15/lib/python3.9/importlib/resources.py", line 147, in files
    return _common.from_package(_get_package(package))
  File "/home/vagrant/.pyenv/versions/3.9.15/lib/python3.9/importlib/_common.py", line 14, in from_package
    return fallback_resources(package.__spec__)
  File "/home/vagrant/.pyenv/versions/3.9.15/lib/python3.9/importlib/_common.py", line 18, in fallback_resources
    package_directory = pathlib.Path(spec.origin).parent
  File "/home/vagrant/.pyenv/versions/3.9.15/lib/python3.9/pathlib.py", line 1082, in __new__
    self = cls._from_parts(args, init=False)
  File "/home/vagrant/.pyenv/versions/3.9.15/lib/python3.9/pathlib.py", line 707, in _from_parts
    drv, root, parts = self._parse_args(args)
  File "/home/vagrant/.pyenv/versions/3.9.15/lib/python3.9/pathlib.py", line 691, in _parse_args
    a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType

I don't know how we'll work around this.