Closed conker84 closed 1 month ago
Could you please post the package name and what command fails for you?
Hi sure:
I executed this from graalpython venv:
venv/bin/pip install transformers
I tried it and I got a different error - it couldn't build tokenizers
dependency because it uses rust. We currently don't support rust packages because Sulong, which we use to execute native dependencies, doesn't support passing rust objects to native libraries (which is necessary to support rust standard library). The issue is being worked by the Sulong team.
Yes I got your same error at first then the CLI suggested upgrading pip so I did
venv/bin/pip install --upgrade pip
and then I got the last error, so I thought that by upgrading pip
the last error was the real one.
oki thanks
Do you know where I can check the progress on this issue? A GitHub issue maybe?
There's no github issue, but I can keep this one updated. But there's more things that will need to happen for this package to work - for example, we currently don't support recent numpy versions that are required by the package. That's also being worked on.
@msimacek any update on this?
The numpy update is almost finished. I have no update from Sulong team on the Rust issue.
Do you know if it is in their (Sulong team) interest to fix it?
Yes, it is. And they are working on it. It's just a very complex change.
thank you so much for the info @msimacek
Hi @msimacek do you have any update on this?
No, still in progress
@msimacek hi! Any progress on this?
Hi, we now support newer numpy, which was one of the items that was blocking this. The rust issue is still unsolved.
@msimacek Hi! Do you know the status of the rust issue? Thank you so much!
We have introduced a new C API backend that uses native execution instead of LLVM on Sulong. That should completely circumvent the issue we were having with passing values to rust. However, the PyO3
framwork that tokenizers
is using currently doesn't work with GraalPy. For start, it literally hardcodes that the interpreter must be named CPython
or PyPy
and fails to start if it's anything else. Surely there will be other issues. @timfel is currently working on creating a patch for PyO3
to work on GraalPy. FYI, I'm currently working on making PyTorch work, which is another prerequisite.
For reference, the PyO3 issue is here: https://github.com/PyO3/pyo3/issues/3052. We also need to update https://github.com/PyO3/maturin, since it has similarly hardcoded PyPy and CPython. I am in the process of fixing those issues, but it's turning into a larger pull request than I anticipated.
With the most recent nightly builds I can use things like GPT-2 or StableDiffusion from Huggingface hub just fine, through transformers, safetensors, diffusers, and torch
Latest relase on Maven Central is 24.0.1. Is it included already? If not ... where to find the nightly builds?
Tried to get it running with 24.0.1 ... but seems numpy is not working in current version ...
Traceback (most recent call last):
File "/home/arne/.local/lib/python3.10/site-packages/numpy/core/__init__.py", line 24, in <module>
from . import multiarray
File "/home/arne/.local/lib/python3.10/site-packages/numpy/core/multiarray.py", line 10, in <module>
from . import overrides
File "/home/arne/.local/lib/python3.10/site-packages/numpy/core/overrides.py", line 8, in <module>
from numpy.core._multiarray_umath import (
ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/graalpy_vfs/proj/hello.py", line 2, in <module>
from transformers import AutoModelForCausalLM
File "/home/arne/.local/lib/python3.10/site-packages/transformers/__init__.py", line 26, in <module>
from . import dependency_versions_check
File "/home/arne/.local/lib/python3.10/site-packages/transformers/dependency_versions_check.py", line 16, in <module>
from .utils.versions import require_version, require_version_core
File "/home/arne/.local/lib/python3.10/site-packages/transformers/utils/__init__.py", line 33, in <module>
from .generic import (
File "/home/arne/.local/lib/python3.10/site-packages/transformers/utils/generic.py", line 28, in <module>
import numpy as np
File "/home/arne/.local/lib/python3.10/site-packages/numpy/__init__.py", line 135, in <module>
raise ImportError(msg) from e
ImportError: Error importing numpy: you should not try to import numpy from
its source directory; please exit the numpy source tree, and relaunch
your python interpreter from there.
Latest relase on Maven Central is 24.0.1. Is it included already? If not ... where to find the nightly builds?
It's not in the 24 release, you would have to use the dev builds from https://github.com/graalvm/graalvm-ce-dev-builds/releases
Tried to get it running with 24.0.1 ... but seems numpy is not working in current version ...
You seem to be importing from /home/arne/.local/lib/python3.10/site-packages/
, this may just be a conflict with CPython 3.10. Please try to install it in a venv (https://www.graalvm.org/latest/reference-manual/python/Python-Runtime/#installing-packages).
Many examples with transformers
now work on GraalPy.
Unfortunately, all the dependencides have to be built from source during the installation, which is slow and requires you to have the right compiler and libraries in the system. Hopefully, in the future we will have prebuilt binary wheels for them.
Is there any plan to support Hugging Face Transformers? They are the next (maybe current) big thing in Python I'm trying to make it work right now but I'm getting the various errors and one of them is:
com.oracle.truffle.api.CompilerDirectives$ShouldNotReachHere: writeByte not implemented