Error with latest importlib-metadata on MacOS

[Linux13524]

When I run audb.load(...) I get an error on MacOS that seems to be related to importlib-metadata from this package here:

Error message

``` Traceback (most recent call last): File "/Library/Python/3.9/site-packages/conans/errors.py", line 34, in conanfile_exception_formatter yield File "/Library/Python/3.9/site-packages/conans/client/conanfile/build.py", line 16, in run_build_method conanfile.build() File "/Users/vagrant/builds/Zq9KZGs5/0/project/devaice/components/cnn10-scene-model/test_package/conanfile.py", line 32, in build self._build_utils.get_reference_data(self.conan_data["model"]["uid"], self.build_folder, get_values_func, None, self.output) File "/tmp/ci-jobs/399674/.conan/data/devaice-build-utils/latest/audeering/develop/export/conanfile.py", line 303, in get_reference_data download_audb_database(entry, build_folder, output=output) File "/tmp/ci-jobs/399674/.conan/data/devaice-build-utils/latest/audeering/develop/export/conanfile.py", line 245, in download_audb_database db = audb.load(entry["database"], version=entry["version"], media=entry["media"], full_path=False, verbose=True) File "/Users/vagrant/Library/Python/3.9/lib/python/site-packages/audb/core/load.py", line 1047, in load db_root = database_cache_root(name, version, cache_root, flavor) File "/Users/vagrant/Library/Python/3.9/lib/python/site-packages/audb/core/cache.py", line 44, in database_cache_root flavor.path(name, version), File "/Users/vagrant/Library/Python/3.9/lib/python/site-packages/audb/core/flavor.py", line 136, in path return os.path.join(name, version, self.short_id) File "/Users/vagrant/Library/Python/3.9/lib/python/site-packages/audb/core/flavor.py", line 148, in short_id return self.id[-8:] File "/Users/vagrant/Library/Python/3.9/lib/python/site-packages/audobject/core/object.py", line 171, in id string = self.to_yaml_s(include_version=False) File "/Users/vagrant/Library/Python/3.9/lib/python/site-packages/audobject/core/object.py", line 337, in to_yaml_s return yaml.dump(self.to_dict(include_version=include_version)) File "/Users/vagrant/Library/Python/3.9/lib/python/site-packages/audobject/core/object.py", line 271, in to_dict name = utils.create_class_key(self.__class__, include_version) File "/Users/vagrant/Library/Python/3.9/lib/python/site-packages/audobject/core/utils.py", line 34, in create_class_key package_names = packages_distributions() File "/Users/vagrant/Library/Python/3.9/lib/python/site-packages/importlib_metadata/__init__.py", line 967, in packages_distributions for pkg in _top_level_declared(dist) or _top_level_inferred(dist): File "/Users/vagrant/Library/Python/3.9/lib/python/site-packages/importlib_metadata/__init__.py", line 1010, in _top_level_inferred opt_names = set(map(_get_toplevel_name, always_iterable(dist.files))) File "/Users/vagrant/Library/Python/3.9/lib/python/site-packages/importlib_metadata/__init__.py", line 511, in files return skip_missing_files( File "/Users/vagrant/Library/Python/3.9/lib/python/site-packages/importlib_metadata/_functools.py", line 102, in wrapper return func(param, *args, **kwargs) File "/Users/vagrant/Library/Python/3.9/lib/python/site-packages/importlib_metadata/__init__.py", line 509, in skip_missing_files return list(filter(lambda path: path.locate().exists(), package_paths)) File "/Users/vagrant/Library/Python/3.9/lib/python/site-packages/importlib_metadata/__init__.py", line 509, in return list(filter(lambda path: path.locate().exists(), package_paths)) File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/pathlib.py", line 1414, in exists self.stat() File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/pathlib.py", line 1222, in stat return self._accessor.stat(self) PermissionError: [Errno 13] Permission denied: '/Library/Python/3.9/site-packages/../../../../var/root/Library/Caches/com.apple.python/Library/Python/3.9/site-packages/virtualenv/__init__.cpython-39.pyc' ```

Python version is 3.9. I cannot reproduce it on Linux unfortunately..

Steps to reproduce:

• Install audb: pip3 install audb

• Check latest importlib-metadata is installed (as from a fresh install of audb): pip3 install -U importlib-metadata

• Run in Python:

  >>> from audb.core.flavor import Flavor
  >>> flavor = Flavor(sampling_rate=16000)
  >>> flavor.path("test", "1.0.0")

• Result is the error above.

When installing older importlib-metadata it works fine:

• pip3 install importlib-metadata==5.0.0

• Run in Python:

  >>> from audb.core.flavor import Flavor
  >>> flavor = Flavor(sampling_rate=16000)
  >>> flavor.path("test", "1.0.0")

• Result is:

  'test/1.0.0/9be30deb'

[hagenw]

Thanks for reporting this. importlib-metadata is in general a nasty package. In theory it is only needed for Python <=3.7, but so far I was not successful in removing it from the dependencies. Maybe it is a good idea to give it another try.

---

Python version is 3.8

Looking at the error messages it seems more likely that you used Python 3.9?

File "/Library/Python/3.9/site-packages/conans/errors.py", line 34

[hagenw]

I had another look importlib.metadata provides the needed packages_distributions() only since Python 3.10, see https://docs.python.org/3/library/importlib.metadata.html#mapping-import-to-distribution-packages, so we have to stay with importlib-metadata.

As it is reported that it works with importlib-metadata==5.0.0, the easiest solution might be to simply require importlib-metadata<6.0.0 in the dependencies. It would also help to add an example to the tests of audobject that also fails for newer versions of importlib-metadata so we don't have to use audb to detect the error. At the moment, all tests are passing with importlib-metadata>=6.8.0, see https://github.com/audeering/audobject/pull/85

[Linux13524]

Looking at the error messages it seems more likely that you used Python 3.9?

Yes, true. Sorry I fixed it above.

At the moment, all tests are passing with importlib-metadata>=6.8.0, see https://github.com/audeering/audobject/pull/85

So I understand it correctly that there is currently no way to test it without audb? And why does it not fail in audb tests?

[hagenw]

We maybe could test it with audb, but my goal would be to write a test here inside audobject that fails with your error, so we don't need to use audb.

[hagenw]

I re-implemented the audb.Flavor related code in a test at https://github.com/audeering/audobject/pull/89, and also updated the audb test under MacOS to use Python 3.9 at https://github.com/audeering/audb/pull/330. None of those tests are failing even thought they both use importlib-metadata==6.8.0.

So maybe the error is related to the particular setup of your MacOS PC/runner?

[Linux13524]

OK strange. I'll try to run the new test on our macOS runner and see if that fails.

[Linux13524]

I cannot reproduce it on a MacBook.. So it seems to be a problem of the Python setup on the macOS runners, maybe due to having multiple Python versions installed with Homebrew..