search

ahupp / python-magic

A python wrapper for libmagic
Other
2.6k stars 280 forks source link

Persistent ImportError: failed to find libmagic #238

Closed fsboehme closed 3 years ago

fsboehme commented 3 years ago

I tried installing libmagic with both Homebrew and Macports (on a new Macbook Air M1 2020), but I'm unable to get it to work. What am I missing?

python-magic==0.4.22

import magic
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File ".../venv/lib/python2.7/site-packages/magic/__init__.py", line 210, in <module>
    libmagic = loader.load_lib()
  File ".../venv/lib/python2.7/site-packages/magic/loader.py", line 39, in load_lib
    raise ImportError('failed to find libmagic.  Check your installation')
ImportError: failed to find libmagic.  Check your installation
% man libmagic

LIBMAGIC(3)              BSD Library Functions Manual              LIBMAGIC(3)
...
BSD                              June 8, 2019                              BSD
cameron-emburse commented 3 years ago

Seems like homebrew has changed to /opt/homebrew/Cellar instead of /usr/local/Cellar/ where this package is looking.

fsboehme commented 3 years ago

Thanks for the quick reply! Combining your answer and this article I found earlier, I've created a symlink here:

cd /usr/local/lib
sudo ln -s /opt/homebrew/Cellar/libmagic/5.39/lib/libmagic.dylib libmagic.dylib

(I first tried only creating a symlink from /usr/local/Cellar/ to /opt/homebrew/Cellar but that didn't seem to do anything)

Now I'm getting

>>> import magic
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".../venv/lib/python2.7/site-packages/magic/__init__.py", line 210, in <module>
    libmagic = loader.load_lib()
  File ".../venv/lib/python2.7/site-packages/magic/loader.py", line 16, in load_lib
    libmagic = ctypes.CDLL(dll)
  File ".../opt/anaconda3/envs/py2/lib/python2.7/ctypes/__init__.py", line 366, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: dlopen(/usr/local/lib/libmagic.dylib, 6): no suitable image found.  Did find:
        /usr/local/lib/libmagic.dylib: mach-o, but wrong architecture
        /opt/homebrew/Cellar/libmagic/5.39/lib/libmagic.1.dylib: mach-o, but wrong architecture

any ideas?

fsboehme commented 3 years ago

I'm guessing this is something. But no idea how to rectify...

% lipo -info /opt/homebrew/Cellar/libmagic/5.39/lib/libmagic.1.dylib                
Non-fat file: /opt/homebrew/Cellar/libmagic/5.39/lib/libmagic.1.dylib is architecture: arm64
% lipo -info ******/venv/bin/python
Non-fat file: /Users/felixboehme/PycharmProjects/leaguelobster/mysite/venv/bin/python is architecture: x86_64
cameron-emburse commented 3 years ago

To be honest, I just modified the python-magic to in my venv point to /opt/homebrew/Cellar/libmagic/*/lib/libmagic.dylib in addition to /usr/local/Cellar/libmagic/*/lib/libmagic.dylib for now...

fsboehme commented 3 years ago

Figured it out. The problem was that, due to compatibility issues with python2 on the M1 macbooks, I've had to install python2 via anaconda, which uses x86_64 architecture. Then I installed libmagic via Homebrew with arm64 architecture.

So the solution was:

  1. Install libmagic through anaconda via conda-forge 
    conda install -c conda-forge libmagic
  2. Then find your anaconda packages folder (search for the pkgs folder) and the libmagic folder within that and create a symlink in usr/local/lib to that libmagic.dylib. Mine looked like this: 
    cd /usr/local/lib
sudo ln -s /Users/******/opt/anaconda3/pkgs/libmagic-5.39-haaf19a9_0/lib/libmagic.dylib libmagic.dylib
  3. import magic works!
cameron-emburse commented 3 years ago

FWIW, I think this is still an issue for 3.whatever apple m1 supports (3.9 in my case)

anentropic commented 3 years ago

this error is still affecting anyone installing on arm mac due to: https://github.com/ahupp/python-magic/blob/5fa9055bdc4159ebab0699ae0429de29ae171267/magic/loader.py#L17

and: https://docs.brew.sh/FAQ#why-should-i-install-homebrew-in-the-default-location

The default prefix is /usr/local for macOS on Intel, /opt/homebrew for macOS on Apple Silicon/ARM, and /home/linuxbrew/.linuxbrew for Linux

I guess could be fixed just by adding extra default search paths

another option could be to detect the platform and parse the actual cellar path from output of brew info --json libmagic

anentropic commented 3 years ago

ignore me... '/opt/homebrew/lib' is searched in current version of python-magic and my homebrew has installed a libmagic.dylib symlink there already

the problem is actually my project is using older version of python-magic 🤦