KeyError: 'sources'

[Schamschula]

When building imagecodecs 2020.12.22 under MacPorts, I get the following error:

Executing: cd "/opt/local/var/macports/build/_Users_marius_Development_MacPorts_ports_python_py-imagecodecs/py38-imagecodecs/work/imagecodecs-2020.12.22" && /opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8 setup.py --no-user-cfg build -j8 Traceback (most recent call last): File "setup.py", line 509, in ext_modules=[extension(name) for name in sorted(EXTENSIONS)], File "setup.py", line 509, in extmodules=[extension(name) for name in sorted(EXTENSIONS)], File "setup.py", line 460, in extension sources=[f'imagecodecs/{name}' + EXT] + ext['sources'], KeyError: 'sources'

It looks like there was a rather significant rewrite of setup.py with this update. I haven't been able to trace the issue, but my guess is that that somehow the key 'sources' doesn't get defined.

[cgohlke]

File "setup.py", line 460, in extension sources=[f'imagecodecs/{name}' + EXT] + ext['sources'],

Looks like you are using a patched version of setup.py. This is line 455, not 460:

https://github.com/cgohlke/imagecodecs/blob/4db82e0beedaf2dbc4b2153cb2a9286eb701186b/setup.py#L450-L463

[Schamschula]

Yes. In order to get it to work under MacPorts I had to do bit of patching:

https://github.com/macports/macports-ports/blob/35d1926b1641627341872e0263e24ac04d39693e/python/py-imagecodecs/files/patch-setup.py.diff

[cgohlke]

I don't see how that patch can apply to the new release. It needs to be adjusted. The relevant change is the usage of ext(...) instead of dict(...).

[Schamschula]

I doesn't. It was just to give you an idea of what I had to do for the previous version. I just didn't have the new version of the patch posted. Here you go: https://gist.github.com/Schamschula/5371841b19b74a3901ea417342021462

[cgohlke]

Is there a unique environment variable or another reliable way to detect if the build is running on MacPorts? I could add those changes to setup.py as a customize_build function:

https://github.com/cgohlke/imagecodecs/blob/4db82e0beedaf2dbc4b2153cb2a9286eb701186b/setup.py#L358-L369

[Schamschula]

Just to see if MacPorts is installed, one can test for port version. However, the user can install MacPorts anywhere, but most do install it in the default /opt/local tree. The easiest way to tell where it is installed is by running locate bin/port in the shell.

[cgohlke]

Could you provide a Python function that returns True when running on MacOS and locate bin/port succeeds? I have no way to test this.

[Schamschula]

To check for macOS

import platform hostOS = platform.system()

it will give Darwin for macOS

The following gets the path to the port executable

import shutil portPath = shutil.which('port')

[cgohlke]

Try v2020.12.24 without patching:

https://github.com/cgohlke/imagecodecs/blob/98a6ab040335393865e40a754c43970ded9aeb9a/setup.py#L357-L393

[Schamschula]

I just tried to build v2020.12.24, but even though I see del EXTENSIONS['jpegxr'] in setup.py, I get

:info:build imagecodecs/_jpegxr.c:617:10: fatal error: 'windowsmediaphoto.h' file not found :info:build #include "windowsmediaphoto.h" :info:build ^~~~~

[cgohlke]

Probably the detection of MacPorts failed. What do os.environ.get('LD_LIBRARY_PATH', os.environ.get('LIBRARY_PATH', '')) and shutil.which('port') return?

[Schamschula]

In [1]: import os In [2]: os.environ.get('LD_LIBRARY_PATH', os.environ.get('LIBRARY_PATH', '')) Out[2]: ''

This is not unexpected, as I'm running this in ipython, not from within the port command. port adds needed environment variables before executing various stages.

In [4]: import shutil In [5]: shutil.which('port') Out[5]: '/opt/local/bin/port'

[Schamschula]

I ended up short circuiting the check for MacPorts, for the time being.

After that, I had to adjust the include path for zopfli.h.

See: https://github.com/macports/macports-ports/commit/cf6402af5977d14b3795acc5e9211f615c326171