Cannot import load_cifar10: `/usr/local/lib/libsndfile.1.dylib` is `x86_64` but `arm64` needed

[RahulBhalley]

Here's the traceback:

Traceback (most recent call last):
  File "/Users/codegod/apple-mlx/mlx-examples/cifar/main.py", line 7, in <module>
    from dataset import get_cifar10
  File "/Users/codegod/apple-mlx/mlx-examples/cifar/dataset.py", line 2, in <module>
    from mlx.data.datasets import load_cifar10
  File "/Users/codegod/apple-mlx/venv/lib/python3.11/site-packages/mlx/data/__init__.py", line 3, in <module>
    from ._c import *
ImportError: dlopen(/Users/codegod/apple-mlx/venv/lib/python3.11/site-packages/mlx/data/_c.cpython-311-darwin.so, 0x0002): Library not loaded: /opt/homebrew/opt/libsndfile/lib/libsndfile.1.dylib
  Referenced from: <9F7752F7-AB7A-3025-A4B7-0EFABE6D194E> /Users/codegod/apple-mlx/venv/lib/python3.11/site-packages/mlx/data/_c.cpython-311-darwin.so
  Reason: tried: '/usr/local/lib/libsndfile.1.dylib' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/usr/local/lib/libsndfile.1.dylib' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/opt/homebrew/opt/libsndfile/lib/libsndfile.1.dylib' (no such file), '/opt/homebrew/opt/libsndfile/lib/libsndfile.1.dylib' (no such file), '/libsndfile.1.dylib' (no such file), '/opt/homebrew/opt/libsndfile/lib/libsndfile.1.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/opt/libsndfile/lib/libsndfile.1.dylib' (no such file), '/opt/homebrew/opt/libsndfile/lib/libsndfile.1.dylib' (no such file)

[dastrobu]

Same here, get the same traceback when trying to import: import mlx.data as dx.

After looking into the docs it says:

The following dependencies are optional but without them some functionality of MLX data will not be built. Below we enumerate the list of dependencies and subsequently we provide example shell commands to install them using homebrew or on Ubuntu using apt.

libsndfile to load audio files.
libsamplerate for audio samplerate conversion.
ffmpeg for loading video files.
libjpegturbo to load JPG images.
zlib, bzip2 and liblzma to process compressed streams.
aws-sdk for accessing S3 buckets.

following dependencies are optional does not seem to be true anymore. Either libsndfile should really be optional, or the error message could be more user friendly. For example: libsndfile could not be found, consider installing it with: brew install libsndfile.

After I installed the dependencies as stated in the docs everything works.

[RahulBhalley]

I did it all but now I'm getting this error:

In [1]: import mlx.data as dx
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Cell In[1], line 1
----> 1 import mlx.data as dx

File ~/venvs/mlx/lib/python3.11/site-packages/mlx/data/__init__.py:3
      1 # Copyright © 2023 Apple Inc.
----> 3 from ._c import *
      4 from ._c import __version__

ImportError: dlopen(/Users/codegod/venvs/mlx/lib/python3.11/site-packages/mlx/data/_c.cpython-311-darwin.so, 0x0002): symbol not found in flat namespace '_jpeg_resync_to_restart'

[RahulBhalley]

I have commented out imports in /Users/codegod/venvs/mlx/lib/python3.11/site-packages/mlx/data/datasets/__init__.py, and as a result, I am able to import mlx.data. However, the functions under mlx.data that are likely linked to C++ are now unavailable. For instance, when I try to use mnist_train = load_mnist(train=True), it throws an error: AttributeError: module 'mlx.data' has no attribute 'buffer_from_vector', and other functions are also not accessible.

[RahulBhalley]

@awni sorry to ping but could you help me?

[awni]

I would not comment the line you commented. Rather it looks like an issue with your Jpeg install. How did you install the dependencies for mlx-data? Could you try reinstalling or upgrading brew install libjpegturbo?

[RahulBhalley]

Already installed.

(mlx) codegod@Rahuls-MacBook-Air mlx-data % brew install jpeg-turbo
Warning: jpeg-turbo 3.0.0 is already installed and up-to-date.
To reinstall 3.0.0, run:
  brew reinstall jpeg-turbo

[RahulBhalley]

1. I installed the latest of Python 3.11 from official website. (https://www.python.org/downloads/release/python-3117/)
2. Then I uninstalled Homebrew with /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/uninstall.sh)"
3. Next, I ran this command: brew install libsndfile libsamplerate ffmpeg jpeg-turbo zlib bzip2 xz aws-sdk-cpp.

Note: If I install Python 3.11 with brew install python@3.11 then running pip3 install mlx says "no compatible distribution found" possibly because brew installs x86 version of Python for me.

Then I ran the following code:

import mlx
import mlx.data as dx

And I get this error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/codegod/venvs/mlx/lib/python3.11/site-packages/mlx/data/__init__.py", line 3, in <module>
    from ._c import *
ImportError: dlopen(/Users/codegod/venvs/mlx/lib/python3.11/site-packages/mlx/data/_c.cpython-311-darwin.so, 0x0002): Library not loaded: /opt/homebrew/opt/libsndfile/lib/libsndfile.1.dylib
  Referenced from: <9F7752F7-AB7A-3025-A4B7-0EFABE6D194E> /Users/codegod/venvs/mlx/lib/python3.11/site-packages/mlx/data/_c.cpython-311-darwin.so
  Reason: tried: '/opt/homebrew/opt/libsndfile/lib/libsndfile.1.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/opt/libsndfile/lib/libsndfile.1.dylib' (no such file), '/opt/homebrew/opt/libsndfile/lib/libsndfile.1.dylib' (no such file)

[RahulBhalley]

The Hombrew's prefix was /usr/local (means it's using Rosetta 2) but it was not pointing to /opt/homebrew (for running as on ARM) even after reinstall.

So, I added eval "$(/opt/homebrew/bin/brew shellenv)" to ~/.zshrc and sourced it. Now MLX data is importing.

PS: Writing here for future reference.