Cannot load dylib on Intel macbooks

[quantumjot]

From Ania:

1.4 GHz Intel Core i5

File /opt/anaconda3/envs/btrack-env/lib/python3.10/site-packages/btrack/libwrapper.py:84, in load_library(filename)
     83 try:
---> 84     lib = ctypes.cdll.LoadLibrary(full_lib_file)
     85     logger.info(f"Loaded btrack: {full_lib_file}")

File /opt/anaconda3/envs/btrack-env/lib/python3.10/ctypes/__init__.py:452, in LibraryLoader.LoadLibrary(self, name)
    451 def LoadLibrary(self, name):
--> 452     return self._dlltype(name)

File /opt/anaconda3/envs/btrack-env/lib/python3.10/ctypes/__init__.py:374, in CDLL.__init__(self, name, mode, handle, use_errno, use_last_error, winmode)
    373 if handle is None:
--> 374     self._handle = _dlopen(self._name, mode)
    375 else:

OSError: dlopen(/opt/anaconda3/envs/btrack-env/lib/python3.10/site-packages/btrack/libs/libtracker.dylib, 6): no suitable image found.  Did find:
    /opt/anaconda3/envs/btrack-env/lib/python3.10/site-packages/btrack/libs/libtracker.dylib: cannot load 'libtracker.dylib' (load command 0x80000034 is unknown)
    /opt/anaconda3/envs/btrack-env/lib/python3.10/site-packages/btrack/libs/libtracker.dylib: cannot load 'libtracker.dylib' (load command 0x80000034 is unknown)

The above exception was the direct cause of the following exception:

OSError                                   Traceback (most recent call last)
Cell In[12], line 1
----> 1 with btrack.BayesianTracker() as tracker:
      2 
      3     # configure the tracker using a config file
      4     tracker.configure(cfg)
      5     tracker.max_search_radius = 20

File /opt/anaconda3/envs/btrack-env/lib/python3.10/site-packages/btrack/core.py:130, in BayesianTracker.__init__(self, verbose)
    128 # load the library, get an instance of the engine
    129 self._initialised = False
--> 130 self._lib = libwrapper.get_library()
    131 self._engine = self._lib.new_interface(verbose)
    133 if not verbose:

File /opt/anaconda3/envs/btrack-env/lib/python3.10/site-packages/btrack/libwrapper.py:94, in get_library()
     92 def get_library():  # noqa: PLR0915
     93     """Loads and returns the btrack shared library."""
---> 94     lib = load_library(os.path.join(BTRACK_PATH, "libs", "libtracker"))
     96     # deal with constructors/destructors
     97     lib.new_interface.restype = ctypes.c_void_p

File /opt/anaconda3/envs/btrack-env/lib/python3.10/site-packages/btrack/libwrapper.py:87, in load_library(filename)
     85     logger.info(f"Loaded btrack: {full_lib_file}")
     86 except OSError as err:
---> 87     raise OSError(f"Cannot load shared library {full_lib_file}") from err
     89 return lib

OSError: Cannot load shared library /opt/anaconda3/envs/btrack-env/lib/python3.10/site-packages/btrack/libs/libtracker.dylib

[paddyroddy]

@p-j-smith it works for you though right?

[p-j-smith]

hmmm, yeah I just made a new test environment and installed btrack from PyPI and the example notebook runs okay. Plus I see the files are where they should be:

-rw-r--r--   1 paul  staff   3.2M  5 Apr 16:43 libtracker.DLL
-rw-r--r--   1 paul  staff   889K  5 Apr 16:43 libtracker.dylib
drwxr-xr-x   5 paul  staff   160B  5 Apr 16:43 .
-rw-r--r--   1 paul  staff   542K  5 Apr 16:43 libtracker.so

I'm also on an Intel Mac, Monterey 12.6.3. Do you know how Ania installed btrack? And the mac os version?

[quantumjot]

Yes - it's weird. It seems to find the file (and it's there!), but doesn't like the symbols: (load command 0x80000034 is unknown)

I've asked for a few more details on the machine.

[ania-m-b]

To answer your questions I am using Mojave 10.14.6. I installed it with both variations one from the installation page and the other from developer guide.

This is the type of error that pops up:

/SDKs/MacOSX10.14.sdk/usr/include/string.h:152: /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/sys/_types/_ssize_t.h:31:9: error: unknown type name 'darwin_ssize_t'; did you mean 'darwin_blksize_t'? typedef darwin_ssize_t ssize_t; ^ /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/sys/_types.h:56:25: note: 'darwin_blksize_t' declared here typedef __int32_t __darwin_blksize_t; / preferred block size / ^ fatal error: too many errors emitted, stopping now [-ferror-limit=] 20 errors generated. make: *** [btrack/obj/bayes.o] Error 1

[paddyroddy]

Hard to tell from looking at that alone, but Eigen (which btrack is built on) requires C++17. I wonder if you are using an older version (Mojave is quite old now). Have you tried brew install g++ and following the appropriate instructions?

Or easier still, update your Mac if possible.

[nthndy]

I've just come across what looks like the same error when updating to v0.5.2

---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
File ~/miniconda3/envs/btrack_dev/lib/python3.9/site-packages/btrack/libwrapper.py:84, in load_library(filename)
     83 try:
---> 84     lib = ctypes.cdll.LoadLibrary(full_lib_file)
     85     logger.info(f"Loaded btrack: {full_lib_file}")

File ~/miniconda3/envs/btrack_dev/lib/python3.9/ctypes/__init__.py:460, in LibraryLoader.LoadLibrary(self, name)
    459 def LoadLibrary(self, name):
--> 460     return self._dlltype(name)

File ~/miniconda3/envs/btrack_dev/lib/python3.9/ctypes/__init__.py:382, in CDLL.__init__(self, name, mode, handle, use_errno, use_last_error, winmode)
    381 if handle is None:
--> 382     self._handle = _dlopen(self._name, mode)
    383 else:

OSError: /home/dayn/miniconda3/envs/btrack_dev/lib/python3.9/site-packages/btrack/libs/libtracker.so: cannot open shared object file: No such file or directory

The above exception was the direct cause of the following exception:

OSError                                   Traceback (most recent call last)
Cell In[3], line 1
----> 1 with btrack.BayesianTracker() as tracker:
      2         # configure the tracker using a config file
      3         tracker.configure(config_fn)

File ~/miniconda3/envs/btrack_dev/lib/python3.9/site-packages/btrack/core.py:130, in BayesianTracker.__init__(self, verbose)
    128 # load the library, get an instance of the engine
    129 self._initialised = False
--> 130 self._lib = libwrapper.get_library()
    131 self._engine = self._lib.new_interface(verbose)
    133 if not verbose:

File ~/miniconda3/envs/btrack_dev/lib/python3.9/site-packages/btrack/libwrapper.py:94, in get_library()
     92 def get_library():  # noqa: PLR0915
     93     """Loads and returns the btrack shared library."""
---> 94     lib = load_library(os.path.join(BTRACK_PATH, "libs", "libtracker"))
     96     # deal with constructors/destructors
     97     lib.new_interface.restype = ctypes.c_void_p

File ~/miniconda3/envs/btrack_dev/lib/python3.9/site-packages/btrack/libwrapper.py:87, in load_library(filename)
     85     logger.info(f"Loaded btrack: {full_lib_file}")
     86 except OSError as err:
---> 87     raise OSError(f"Cannot load shared library {full_lib_file}") from err
     89 return lib

OSError: Cannot load shared library /home/dayn/miniconda3/envs/btrack_dev/lib/python3.9/site-packages/btrack/libs/libtracker.so

[nthndy]

am running on ubuntu 20.04 Intel Core i9

[paddyroddy]

I've just tried on a ubuntu Intel image (with the platform because I'm on ARM)

docker run -it --rm --platform=linux/x86_64 python:3.10 sh

And was able to install btrack. @quantumjot is there an easy test to make sure the C++ is installed?

[quantumjot]

No easy test, but a couple of thoughts:

• run the pytest suite locally, or
• import btrack and create an instance of BayesianTracker - that should force import of the shared library

[paddyroddy]

Okay so

docker run -it --rm --platform=linux/x86_64 python:3.10 sh

Then within the container

pip install btrack ipython

Then start ipython and run

import btrack
btrack.BayesianTracker()

Fails with Cannot load shared library /usr/local/lib/python3.10/site-packages/btrack/libs/libtracker.so. Problem is in local testing is that I have an ARM Mac which we've not built for. It works for @p-j-smith locally. Will investigate.

[paddyroddy]

Can you try with v0.5.11 @nthndy, @ania-m-b, @quantumjot, @p-j-smith?

[paddyroddy]

Can you try with v0.5.11 @nthndy, @ania-m-b, @quantumjot, @p-j-smith?

False alarm. From the docker I can see this hasn't worked.

[ania-m-b]

Hard to tell from looking at that alone, but Eigen (which btrack is built on) requires C++17. I wonder if you are using an older version (Mojave is quite old now). Have you tried brew install g++ and following the appropriate instructions?

Or easier still, update your Mac if possible.

Hi! I have done this previously brew install g++ when installing a different package and I wanted to avoid upgrading to the next MacOS. I am able to run btrack up to version 5.0.

[paddyroddy]

Okay, can everyone try btrack==0.5.13? Note the binaries are built with ubuntu=20.04 and/or macos-11. I'm afraid if you have earlier than that then it might not work.

[dstansby]

I went down a self inflicted sort-of-related rabbit hole this evening (courtesy of @paddyroddy 😉), and while I was down there at one point I was getting errors like:

[1/2] Compiling C++ object tracker.p/tracker.cc.o
FAILED: tracker.p/tracker.cc.o 
ccache c++ -Itracker.p -I. -I.. -I../../include -I../../../include -I/opt/homebrew/Cellar/eigen/3.4.0_1/include/eigen3 -fcolor-diagnostics -Wall -Winvalid-pch -O0 -g -std=c++17 '-arch arm64' -MD -MQ tracker.p/tracker.cc.o -MF tracker.p/tracker.cc.o.d -o tracker.p/tracker.cc.o -c ../tracker.cc
clang: warning: argument unused during compilation: '-arch arm64' [-Wunused-command-line-argument]
In file included from ../tracker.cc:17:
../../include/tracker.h:268:20: error: 'path' is unavailable: introduced in macOS 10.15
  std::filesystem::path m_debug_filepath;

I think this means btrack is using c++ calls only available in macOS >= 10.15, so it won't work with macOS < 10.15.

[quantumjot]

I think this means btrack is using c++ calls only available in macOS >= 10.15, so it won't work with macOS < 10.15.

Interesting! Well this bit of the code base is really only for debugging, so we could remove/refactor this one line for improved compatibility.

[quantumjot]

Also @paddyroddy - I just tried a fresh install (btrack 0.5.13 from pypi) on:

• Intel® Core™ i9-10940X CPU
• Ubuntu 20.04.5 LTS
• Python 3.9

...and everything seemed to work ok.

[paddyroddy]

Fixed in #300.