search

python-pillow / Pillow

Python Imaging Library (Fork)
https://python-pillow.org
Other
12.27k stars 2.23k forks source link

Symbol not found: _jpeg_resync_to_restart #5143

Closed Animenosekai closed 3 years ago

Animenosekai commented 3 years ago

What did you do?

Pillow installs successfully, but doesn't want to be imported. It's actually been about 5 hours that I'm trying multiple things to get Pillow working. I actually tried to reinstall Pillow without jpeg by doing pip install Pillow --global-option="build_ext" --global-option="--disable-jpeg" and it did work for a time but: 1 - Without jpeg, it becomes quickly hard to go further 2 - I tried to reinstall it and now even without jpeg, python throws me a new error : ImportError: dynamic module does not define module export function (PyInit__imaging) Also I tried to build it from source and with different wheels by changing their name which didn't seem to work better.

Apart from libwebp and libfreetype (which aren't on Homebrew), everything is downloaded:

Warning: jpeg 9d is already installed and up-to-date
To reinstall 9d, run `brew reinstall jpeg`
Warning: zlib 1.2.11 is already installed and up-to-date
To reinstall 1.2.11, run `brew reinstall zlib`
Warning: libtiff 4.2.0 is already installed and up-to-date
To reinstall 4.2.0, run `brew reinstall libtiff`
Warning: little-cms 1.19_1 is already installed and up-to-date
To reinstall 1.19_1, run `brew reinstall little-cms`
Warning: openjpeg 2.3.1 is already installed and up-to-date
To reinstall 2.3.1, run `brew reinstall openjpeg`
Warning: libimagequant 2.13.1 is already installed and up-to-date
To reinstall 2.13.1, run `brew reinstall libimagequant`
Warning: libraqm 0.7.1 is already installed and up-to-date
To reinstall 0.7.1, run `brew reinstall libraqm`
Warning: libxcb 1.14 is already installed and up-to-date
To reinstall 1.14, run `brew reinstall libxcb`

Also something to take into account (maybe) is that I'm on a MacBook Air with the new M1 chip (Apple Silicon/ARM)

What did you expect to happen?

To be imported in my projects (more precisely to be imported by the ImageHash module)

What actually happened?

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/animenosekai/miniforge3/lib/python3.9/site-packages/PIL/Image.py", line 94, in <module>
    from . import _imaging as core
ImportError: dlopen(/Users/animenosekai/miniforge3/lib/python3.9/site-packages/PIL/_imaging.cpython-39-darwin.so, 2): Symbol not found: _jpeg_resync_to_restart
  Referenced from: /Users/animenosekai/miniforge3/lib/python3.9/site-packages/PIL/_imaging.cpython-39-darwin.so
  Expected in: flat namespace
 in /Users/animenosekai/miniforge3/lib/python3.9/site-packages/PIL/_imaging.cpython-39-darwin.so

What are your OS, Python and Pillow versions?

from PIL import Image

If you need any further information I'm obviously here to do my best to help you

radarhere commented 3 years ago

Hi. Yeah, I wouldn't try going down the road of installing without jpeg.

Searching around, there seem to be a variety of solutions suggested for this.

https://github.com/python-pillow/Pillow/issues/543 - make sure you don't have PIL and Pillow installed http://www.brambraakman.com/blog/comments/installing_pil_in_snow_leopard_jpeg_resync_to_restart_error/ - are you using an older GCC? After looking at https://stackoverflow.com/questions/6149634/imaging-c-module-error-in-python-pil, I'm wondering - if you uninstall libjpeg with brew, does it actually make a difference? If it doesn't, then it would seem that Pillow is detecting libjpeg in a different location.

Animenosekai commented 3 years ago

@radarhere Thank's for replying!

I made sure that I didn't have PIL installed (with pip list) and PIL doesn't seem to be installed (I only see Pillow)

GCC doesn't seem to be an older GCC, here is the result of gcc -v:

Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/4.2.1
Apple clang version 12.0.0 (clang-1200.0.32.28)
Target: arm64-apple-darwin20.1.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

Uninstalling libjpeg with brew (brew uninstall libjpeg --ignore-dependencies) doesn't seem to make any difference: If Pillow is detecting libjpeg in a different location, what should I do?

Animenosekai commented 3 years ago

I just realised while reinstalling libjpeg that brew gave me warnings about the formula not being symlinked into /usr/local.

I'm pretty sure this is the error.

The formula built, but is not symlinked into /usr/local
Could not symlink bin/cjpeg
Target /usr/local/bin/cjpeg
already exists. You may want to remove it:
  rm '/usr/local/bin/cjpeg'

To force the link and overwrite all conflicting files:
  brew link --overwrite jpeg

To list all files that would be deleted:
  brew link --overwrite --dry-run jpeg

Should I overwrite them?

Would remove:
/usr/local/bin/cjpeg
/usr/local/bin/djpeg
/usr/local/bin/jpegtran
/usr/local/bin/rdjpgcom
/usr/local/bin/wrjpgcom
/usr/local/include/jconfig.h
/usr/local/include/jerror.h
/usr/local/include/jmorecfg.h
/usr/local/include/jpeglib.h
/usr/local/share/man/man1/cjpeg.1
/usr/local/share/man/man1/djpeg.1
/usr/local/share/man/man1/jpegtran.1
/usr/local/share/man/man1/rdjpgcom.1
/usr/local/share/man/man1/wrjpgcom.1
/usr/local/lib/libjpeg.a
/usr/local/lib/libjpeg.dylib -> /usr/local/lib/libjpeg.8.dylib
radarhere commented 3 years ago

Those files don't exist on my copy of Big Sur, so they would have been something you installed. If you're happy to remove them, you can.

Animenosekai commented 3 years ago 
brew link --overwrite jpeg

Linking /usr/local/Cellar/jpeg/9d... 18 symlinks created

I then uninstalled Pillow and reinstalled it without any cache with pip pip uninstall Pillow --> y; pip install Pillow --no-cache-dir

The message seems to have changed when I try to import Image from PIL import Image

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/animenosekai/miniforge3/lib/python3.9/site-packages/PIL/Image.py", line 94, in <module>
    from . import _imaging as core
ImportError: dynamic module does not define module export function (PyInit__imaging)

I then tried to uninstall and reinstall Pillow

Seems to have the same problem as before now:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/animenosekai/miniforge3/lib/python3.9/site-packages/PIL/Image.py", line 94, in <module>
    from . import _imaging as core
ImportError: dlopen(/Users/animenosekai/miniforge3/lib/python3.9/site-packages/PIL/_imaging.cpython-39-darwin.so, 2): Symbol not found: _jpeg_resync_to_restart
  Referenced from: /Users/animenosekai/miniforge3/lib/python3.9/site-packages/PIL/_imaging.cpython-39-darwin.so
  Expected in: flat namespace
 in /Users/animenosekai/miniforge3/lib/python3.9/site-packages/PIL/_imaging.cpython-39-darwin.so

Does this mean that when Pillow is installed from a wheel (cached when simply pip install Pillow) it gives me the _jpeg_resync_to_restart error and when building it from the source code (when using --no-cache-dir or using a --global-option flag) it gives me the dynamic module does not define module export function (PyInit__imaging) error...

radarhere commented 3 years ago

Hmm. I found https://github.com/labsyspharm/mcmicro/issues/86 for 'dynamic module does not define module export function (PyInit__imaging)', but that doesn't seem very helpful.

Here's a zip of a wheel that I compiled on my machine, just in case that is helpful - Pillow-8.0.1-cp39-cp39-macosx_11_0_arm64.whl.zip

Animenosekai commented 3 years ago

Tried to uninstall Pillow and reinstall it with your wheel.

The install was successful but I get another error message when importing Pillow with python (from PIL import Image):

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/animenosekai/miniforge3/lib/python3.9/site-packages/PIL/Image.py", line 94, in <module>
    from . import _imaging as core
ImportError: dlopen(/Users/animenosekai/miniforge3/lib/python3.9/site-packages/PIL/_imaging.cpython-39-darwin.so, 2): Library not loaded: /opt/local/lib/libjpeg.9.dylib
  Referenced from: /Users/animenosekai/miniforge3/lib/python3.9/site-packages/PIL/_imaging.cpython-39-darwin.so
  Reason: no suitable image found.  Did find:
    /usr/local/lib/libjpeg.9.dylib: mach-o, but wrong architecture
    /usr/local/Cellar/jpeg/9d/lib/libjpeg.9.dylib: mach-o, but wrong architecture

Is it because python3 is built for ARM and libjpeg is built for x86_64 ???

Animenosekai commented 3 years ago

Update:

I uninstalled my x86_64 version of libjpeg and installed an ARM version (by reinstalling brew entirely in /opt/local). After trying to install libjpeg (because I needed to recreate all of the symlinks myself since they seemed to have installed into /opt/homebrew while Python was giving an error about libjpeg not found and I could see that it was searching in /opt/local)

I get this error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/animenosekai/miniforge3/lib/python3.9/site-packages/PIL/Image.py", line 94, in <module>
    from . import _imaging as core
ImportError: dlopen(/Users/animenosekai/miniforge3/lib/python3.9/site-packages/PIL/_imaging.cpython-39-darwin.so, 2): Library not loaded: /opt/local/lib/libjpeg.9.dylib
  Referenced from: /Users/animenosekai/miniforge3/lib/python3.9/site-packages/PIL/_imaging.cpython-39-darwin.so
  Reason: no suitable image found.  Did find:
    /opt/local/lib/libjpeg.9.dylib: file too short
    /opt/local/lib/libjpeg.9.dylib: file too short

What does that even mean "file too short" lmao

radarhere commented 3 years ago

Trying again. Hopefully this one doesn't go looking for /opt, Pillow-8.0.1-cp39-cp39-macosx_11_0_arm64.whl 2.zip

Animenosekai commented 3 years ago

Oh wait, it actually worked!!!!

(base) animenosekai@Anises-MacBook Downloads % python3
Python 3.9.1 | packaged by conda-forge | (default, Dec  9 2020, 01:07:47) 
[Clang 11.0.0 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
〜Welcome back to the Python Interpreter〜
>>> from PIL import Image
>>>

No error message

radarhere commented 3 years ago

The second wheel I provided worked? Excellent.

Animenosekai commented 3 years ago

Hope that it will stay working... I'll close the issue now and I'll reopen if anything goes wrong.

Thank you so much @radarhere !!!

radarhere commented 3 years ago

I'll point out here that since the release of Pillow 8.1.1, you should now be able to install Pillow normally again, as we have M1 wheels now.

abrigard commented 3 years ago

I tried to install the default version of Pillow (8.2.0) and did not work. I had the same errors. I forced 8.0.1 and worked ok.

radarhere commented 3 years ago

@abrigard if you'd like us to try and help you debug that, please create a new issue

jwcaceres78 commented 3 years ago

Trying again. Hopefully this one doesn't go looking for /opt, Pillow-8.0.1-cp39-cp39-macosx_11_0_arm64.whl 2.zip

Hi @radarhere, I am in macOS 11.5.2 (with M1 processor), but I have this error: ERROR: Pillow-8.0.1-cp39-cp39-macosx_11_0_arm64.whl is not a supported wheel on this platform.

radarhere commented 3 years ago

Hi. Instead of trying my old wheel from here, you can try installing from our latest release on PyPI - https://pypi.org/project/Pillow/. Here's a direct link to the wheel if that's helpful - https://files.pythonhosted.org/packages/c7/bd/a3277af75cc35f5ce73807424fead64c397fd9af1e915de9e741e5a1a72e/Pillow-8.3.1-cp39-cp39-macosx_11_0_arm64.whl

What version of Python are you using?

jwcaceres78 commented 3 years ago

Hi. Instead of trying my old wheel from here, you can try installing from our latest release on PyPI - https://pypi.org/project/Pillow/. Here's a direct link to the wheel if that's helpful - https://files.pythonhosted.org/packages/c7/bd/a3277af75cc35f5ce73807424fead64c397fd9af1e915de9e741e5a1a72e/Pillow-8.3.1-cp39-cp39-macosx_11_0_arm64.whl

What version of Python are you using?

Hi, I tried with the above file and have the same error, I am using Python 3.8.2

radarhere commented 3 years ago

Ah, yep, ok, so both of those links I gave were for Python 3.9. Try this instead - https://files.pythonhosted.org/packages/73/2d/9d345f4b7e47939f80dc0eec4c813d251b81ce7ec0b3bfcb12327ea73788/Pillow-8.3.1-cp38-cp38-macosx_11_0_arm64.whl

jwcaceres78 commented 3 years ago

Ah, yep, ok, so both of those links I gave were for Python 3.9. Try this instead - https://files.pythonhosted.org/packages/73/2d/9d345f4b7e47939f80dc0eec4c813d251b81ce7ec0b3bfcb12327ea73788/Pillow-8.3.1-cp38-cp38-macosx_11_0_arm64.whl

Unfortunately I have the same error :(, I am very frustrated with this error.

radarhere commented 3 years ago

Did you want to upgrade pip and then try again? python3.8 -m pip install --upgrade pip

pekanboy commented 3 months ago

I have so problem MacOs M2 Python 3.7.12

radarhere commented 3 months ago

Could you provide some more information? What version of Pillow are you using?

WernerResponse commented 3 months ago

Hi, same problem here. MacBook with M1 processor. Running Sonoma 14.5 Using arcade with pillow 9.3.0 as newer versions of pillow are not supported by arcade. Python 3.12.4 Only pillow installed not PIL. gcc version: Apple clang version 15.0.0 (clang-1500.3.9.4) Target: arm64-apple-darwin23.5.0

doing pip list Package Version

arcade 2.6.17 attrs 23.2.0 cffi 1.16.0 Pillow 9.3.0 pip 24.2 pycparser 2.22 pyglet 2.0.dev23 pymunk 6.4.0 pytiled-parser 2.2.0 typing_extensions 4.12.2

(No PIL there)

Looking in .venv lib directory there is a directory called PIL in there as well as Pillow 9.3.0.

Error message: ImportError: dlopen(/Users/..../.venv/lib/python3.12/site-packages/PIL/_imaging.cpython-312-darwin.so, 0x0002): symbol not found in flat namespace '_jpeg_resync_to_restart'

Removing PIL from .venv directory gives me this error: ModuleNotFoundError: No module named 'PIL'

So I guess it is not a good idea to remove that directory...

I'm lost. Can you help me?

radarhere commented 3 months ago

You can see from https://pillow.readthedocs.io/en/stable/installation/python-support.html that Pillow 9.3.0 doesn't support Python 3.12. If you must use Pillow 9.3.0, perhaps downgrading to Python 3.11 would fix the matter? That would allow you use a prebuilt wheel.

pekanboy commented 3 months ago

Could you provide some more information? What version of Pillow are you using?

I am use Pillow 8 4 0

WernerResponse commented 3 months ago

You can see from https://pillow.readthedocs.io/en/stable/installation/python-support.html that Pillow 9.3.0 doesn't support Python 3.12. If you must use Pillow 9.3.0, perhaps downgrading to Python 3.11 would fix the matter? That would allow you use a prebuilt wheel.

Using Python 3.11 solved my issue. Thanks. Never thought about that.

radarhere commented 3 months ago

I am use Pillow 8 4 0

How did you install Pillow? Could you try upgrading pip and then installing Pillow again? Could you provide the full traceback of your error message?

As an aside, I naturally recommend that you use the latest version of Pillow, which would involve you upgrading to at least Python 3.8.