Closed Jackenmen closed 1 year ago
@jack1142 just to make sure does your manually build python package include the C++ version?
import os
os.environ["RAPIDFUZZ_IMPLEMENTATION"] = "cpp"
import rapidfuzz
does this work?
I think this might need a fix inside the rapidfuzz library similar to: https://github.com/arvidn/libtorrent/blob/RC_2_0/CMakeLists.txt#L649 which appears to help with a bug in the gcc compiler: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81358
just to make sure does your manually build python package include the C++ version?
Huh, apparently not...
pi@raspberrypi:~ $ python3 -m venv manual_build
pi@raspberrypi:~ $ . manual_build/bin/activate
(manual_build) pi@raspberrypi:~ $ python -m pip install -U rapidfuzz --no-binary :all:
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting rapidfuzz
Using cached rapidfuzz-2.4.2.tar.gz (1.1 MB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
Collecting jarowinkler<2.0.0,>=1.2.0
Using cached jarowinkler-1.2.0.tar.gz (84 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
Building wheels for collected packages: rapidfuzz, jarowinkler
Building wheel for rapidfuzz (PEP 517) ... done
Created wheel for rapidfuzz: filename=rapidfuzz-2.4.2-py3-none-any.whl size=42562 sha256=3b485c4bdd078cfd51a2495371daa7260217043026d717c7a750086a8ce6e57d
Stored in directory: /home/pi/.cache/pip/wheels/af/99/46/b4f077eb37c0b9b373ef6a18e861eb7fad8def017b50f95f8d
Building wheel for jarowinkler (PEP 517) ... done
Created wheel for jarowinkler: filename=jarowinkler-1.2.0-py3-none-any.whl size=6729 sha256=32acf0183fd8c654d3b43a6b3e40798980dadfd07c3b1c7744e3283c52a012cc
Stored in directory: /home/pi/.cache/pip/wheels/e8/9a/b5/479eb2f7259ba5d0c17fe12140829c71ccef2b16c3ba77a537
Successfully built rapidfuzz jarowinkler
Installing collected packages: jarowinkler, rapidfuzz
Successfully installed jarowinkler-1.2.0 rapidfuzz-2.4.2
(manual_build) pi@raspberrypi:~ $ cat repro.py
import os
os.environ["RAPIDFUZZ_IMPLEMENTATION"] = "cpp"
import rapidfuzz
(manual_build) pi@raspberrypi:~ $ python repro.py
Traceback (most recent call last):
File "/home/pi/repro.py", line 3, in <module>
import rapidfuzz
File "/home/pi/manual_build/lib/python3.9/site-packages/rapidfuzz/__init__.py", line 8, in <module>
from rapidfuzz import process, distance, fuzz, string_metric, utils
File "/home/pi/manual_build/lib/python3.9/site-packages/rapidfuzz/process.py", line 4, in <module>
from rapidfuzz.utils import _fallback_import
File "/home/pi/manual_build/lib/python3.9/site-packages/rapidfuzz/utils.py", line 29, in <module>
default_process = _fallback_import("rapidfuzz.utils", "default_process")
File "/home/pi/manual_build/lib/python3.9/site-packages/rapidfuzz/utils.py", line 12, in _fallback_import
mod = importlib.import_module(module + "_cpp")
File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named 'rapidfuzz.utils_cpp'
(manual_build) pi@raspberrypi:~ $
Huh, apparently not...
I honestly didn't expect that rapidfuzz would just continue on with pure Python build when the build fails since most packages don't do that :smile: I guess I should have expected something is wrong since I didn't even have to install python3-dev
package :P
Anyway, now that I've installed python3-dev
and cmake
, I ran another install and that in fact caused me to run into the exact same error:
(manual_build) pi@raspberrypi:~ $ python repro.py
Traceback (most recent call last):
File "/home/pi/repro.py", line 3, in <module>
import rapidfuzz
File "/home/pi/manual_build/lib/python3.9/site-packages/rapidfuzz/__init__.py", line 8, in <module>
from rapidfuzz import process, distance, fuzz, string_metric, utils
File "/home/pi/manual_build/lib/python3.9/site-packages/rapidfuzz/process.py", line 10, in <module>
cdist = _fallback_import("rapidfuzz.process_cdist", "cdist")
File "/home/pi/manual_build/lib/python3.9/site-packages/rapidfuzz/utils.py", line 12, in _fallback_import
mod = importlib.import_module(module + "_cpp")
File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/home/pi/manual_build/lib/python3.9/site-packages/rapidfuzz/process_cdist_cpp.py", line 3, in <module>
from rapidfuzz.process_cdist_cpp_impl import (
ImportError: /home/pi/manual_build/lib/python3.9/site-packages/rapidfuzz/process_cdist_cpp_impl.cpython-39-arm-linux-gnueabihf.so: undefined symbol: __atomic_compare_exchange_8
Looks like this is indeed something that needs to be addressed by rapidfuzz.
I honestly didn't expect that rapidfuzz would just continue on with pure Python build when the build fails since most packages don't do that
Yes this is mostly done to have the library working on all systems. E.g. https://github.com/jamesturk/jellyfish is another text matching library that does the same.
Looks like this is indeed something that needs to be addressed by rapidfuzz.
Could you try to build from the latest main version?
git clone --recursive https://github.com/maxbachmann/RapidFuzz.git
cd RapidFuzz
pip install .
I think this should fix the error.
Could you try to build from the latest main version?
Works fine :+1:
Here's the full build log: build.log
Just tried installing the released rapidfuzz 2.4.3 that's also already available on piwheels and it works fine. Looks like this can be closed, thanks for the fix!
Package name
rapidfuzz
Package version
2.1.4 - 2.4.2
PyPI URL
https://pypi.org/project/rapidfuzz/
piwheels URL
https://www.piwheels.org/project/rapidfuzz/
Python version
I am the maintainer
More information
On freshly flashed Raspberry Pi OS 10 (Legacy) and Raspberry Pi OS 11 armhf, this can be reproduced by running these commands:
Which results in output:
When not using the built wheels, this error does not occur:
I have also tried using
--no-binary rapidfuzz
instead of--no-binary :all:
and in that case, there are also no errors.At first, I thought that this is probably an issue with the package but since I tried installing it without using pre-built wheels and it worked fine, I figured that maybe it has something to do with how piwheels built the package.