snorkel-team / snorkel

A system for quickly generating training data with weak supervision
https://snorkel.org
Apache License 2.0
5.81k stars 857 forks source link

Dependency need update #1718

Closed yinxiangshi closed 1 year ago

yinxiangshi commented 1 year ago

Issue description

On anaconda, snorkel requires numpy=1.19.5, on Apple m chip, NumPy in such version can not work. I am testing if snorkel works with apple metal GPU acceleration.

Code example/repro steps

import numpy
Traceback (most recent call last):
  File "/Users/jhj/anaconda3/envs/snorkel/lib/python3.9/site-packages/numpy/core/__init__.py", line 22, in <module>
    from . import multiarray
  File "/Users/jhj/anaconda3/envs/snorkel/lib/python3.9/site-packages/numpy/core/multiarray.py", line 12, in <module>
    from . import overrides
  File "/Users/jhj/anaconda3/envs/snorkel/lib/python3.9/site-packages/numpy/core/overrides.py", line 7, in <module>
    from numpy.core._multiarray_umath import (
ImportError: dlopen(/Users/jhj/anaconda3/envs/snorkel/lib/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-darwin.so, 0x0002): Library not loaded: '@rpath/libcblas.3.dylib'
  Referenced from: '/Users/jhj/anaconda3/envs/snorkel/lib/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-darwin.so'
  Reason: tried: '/Users/jhj/anaconda3/envs/snorkel/lib/libcblas.3.dylib' (no such file), '/Users/jhj/anaconda3/envs/snorkel/lib/libcblas.3.dylib' (no such file), '/Users/jhj/anaconda3/envs/snorkel/lib/python3.9/site-packages/numpy/core/../../../../libcblas.3.dylib' (no such file), '/Users/jhj/anaconda3/envs/snorkel/lib/libcblas.3.dylib' (no such file), '/Users/jhj/anaconda3/envs/snorkel/lib/libcblas.3.dylib' (no such file), '/Users/jhj/anaconda3/envs/snorkel/lib/python3.9/site-packages/numpy/core/../../../../libcblas.3.dylib' (no such file), '/Users/jhj/anaconda3/envs/snorkel/bin/../lib/libcblas.3.dylib' (no such file), '/Users/jhj/anaconda3/envs/snorkel/bin/../lib/libcblas.3.dylib' (no such file), '/usr/local/lib/libcblas.3.dylib' (no such file), '/usr/lib/libcblas.3.dylib' (no such file)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/jhj/anaconda3/envs/snorkel/lib/python3.9/site-packages/numpy/__init__.py", line 140, in <module>
    from . import core
  File "/Users/jhj/anaconda3/envs/snorkel/lib/python3.9/site-packages/numpy/core/__init__.py", line 48, in <module>
    raise ImportError(msg)
ImportError:

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.9 from "/Users/jhj/anaconda3/envs/snorkel/bin/python"
  * The NumPy version is: "1.19.5"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: dlopen(/Users/jhj/anaconda3/envs/snorkel/lib/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-darwin.so, 0x0002): Library not loaded: '@rpath/libcblas.3.dylib'
  Referenced from: '/Users/jhj/anaconda3/envs/snorkel/lib/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-darwin.so'
  Reason: tried: '/Users/jhj/anaconda3/envs/snorkel/lib/libcblas.3.dylib' (no such file), '/Users/jhj/anaconda3/envs/snorkel/lib/libcblas.3.dylib' (no such file), '/Users/jhj/anaconda3/envs/snorkel/lib/python3.9/site-packages/numpy/core/../../../../libcblas.3.dylib' (no such file), '/Users/jhj/anaconda3/envs/snorkel/lib/libcblas.3.dylib' (no such file), '/Users/jhj/anaconda3/envs/snorkel/lib/libcblas.3.dylib' (no such file), '/Users/jhj/anaconda3/envs/snorkel/lib/python3.9/site-packages/numpy/core/../../../../libcblas.3.dylib' (no such file), '/Users/jhj/anaconda3/envs/snorkel/bin/../lib/libcblas.3.dylib' (no such file), '/Users/jhj/anaconda3/envs/snorkel/bin/../lib/libcblas.3.dylib' (no such file), '/usr/local/lib/libcblas.3.dylib' (no such file), '/usr/lib/libcblas.3.dylib' (no such file)

System info

Additional context

I did the following test:

install NumPy before installing the snorkel, the NumPy can work.

After installing the snorkel, the NumPy will downgrade to 1.19.5, then the NumPy can not work.

yinxiangshi commented 1 year ago

I will try to solve this issue, and test for metal GPU acceleration.

yinxiangshi commented 1 year ago

@rjurney Do u wanna take a look?

github-actions[bot] commented 1 year ago

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 7 days.