search

ashvardanian / SimSIMD

Up to 200x Faster Dot Products & Similarity Metrics — for Python, Rust, C, JS, and Swift, supporting f64, f32, f16 real & complex, i8, and bit vectors using SIMD for both AVX2, AVX-512, NEON, SVE, & SVE2 📐
https://ashvardanian.com/posts/simsimd-faster-scipy/
Apache License 2.0
988 stars 59 forks source link

Bug: source code install fails in compile step with missing simsimd/simsimd.h file #196

Closed jimthompson5802 closed 1 month ago

jimthompson5802 commented 1 month ago

Describe the bug

With a source code install of the package, the install process fails with the following message:

      gcc -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DSIMSIMD_NATIVE_F16=0 -DSIMSIMD_NATIVE_BF16=0 -DSIMSIMD_DYNAMIC_DISPATCH=1 -DSIMSIMD_TARGET_NEON=1 -DSIMSIMD_TARGET_SVE=1 -DSIMSIMD_TARGET_SVE2=1 -DSIMSIMD_TARGET_HASWELL=1 -DSIMSIMD_TARGET_SKYLAKE=1 -DSIMSIMD_TARGET_ICE=1 -DSIMSIMD_TARGET_GENOA=1 -DSIMSIMD_TARGET_SAPPHIRE=1 -Iinclude -I/usr/local/include/python3.11 -c c/lib.c -o build/temp.linux-x86_64-cpython-311/c/lib.o -std=c11 -O3 -ffast-math -fdiagnostics-color=always -w -fopenmp
      c/lib.c:42:10: fatal error: simsimd/simsimd.h: No such file or directory
         42 | #include <simsimd/simsimd.h>
            |          ^~~~~~~~~~~~~~~~~~~
      compilation terminated.
      error: command '/usr/bin/gcc' failed with exit code 1

Steps to reproduce

  1. Download source code distribution file from PyPi 
    wget https://files.pythonhosted.org/packages/89/68/e7cac967435022636d8d545ae435e89bb3f9268bbc2f32cb66ef9cffcd2f/simsimd-5.4.3.tar.gz
  2. Pip install of the downloaded distribution file 
    pip install simsimd-5.4.3.tar.gz

This generates the following output

Processing ./simsimd-5.4.3.tar.gz
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: simsimd
  Building wheel for simsimd (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for simsimd (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [23 lines of output]
      running bdist_wheel
      running build
      running build_py
      running egg_info
      writing simsimd.egg-info/PKG-INFO
      writing dependency_links to simsimd.egg-info/dependency_links.txt
      writing top-level names to simsimd.egg-info/top_level.txt
      reading manifest file 'simsimd.egg-info/SOURCES.txt'
      adding license file 'LICENSE'
      writing manifest file 'simsimd.egg-info/SOURCES.txt'
      creating build/lib.linux-x86_64-cpython-311/simsimd
      copying python/annotations/__init__.pyi -> build/lib.linux-x86_64-cpython-311/simsimd
      copying python/annotations/py.typed -> build/lib.linux-x86_64-cpython-311/simsimd
      running build_ext
      building 'simsimd' extension
      creating build/temp.linux-x86_64-cpython-311/c
      creating build/temp.linux-x86_64-cpython-311/python
      gcc -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DSIMSIMD_NATIVE_F16=0 -DSIMSIMD_NATIVE_BF16=0 -DSIMSIMD_DYNAMIC_DISPATCH=1 -DSIMSIMD_TARGET_NEON=1 -DSIMSIMD_TARGET_SVE=1 -DSIMSIMD_TARGET_SVE2=1 -DSIMSIMD_TARGET_HASWELL=1 -DSIMSIMD_TARGET_SKYLAKE=1 -DSIMSIMD_TARGET_ICE=1 -DSIMSIMD_TARGET_GENOA=1 -DSIMSIMD_TARGET_SAPPHIRE=1 -Iinclude -I/usr/local/include/python3.11 -c c/lib.c -o build/temp.linux-x86_64-cpython-311/c/lib.o -std=c11 -O3 -ffast-math -fdiagnostics-color=always -w -fopenmp
      c/lib.c:42:10: fatal error: simsimd/simsimd.h: No such file or directory
         42 | #include <simsimd/simsimd.h>
            |          ^~~~~~~~~~~~~~~~~~~
      compilation terminated.
      error: command '/usr/bin/gcc' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for simsimd
Failed to build simsimd

Extracting the contents of the downloaded simsimd-5.4.3.tar.gz we see that the include sub-directory with the header files is missing.

tar -xzvf simsimd-5.4.3.tar.gz
simsimd-5.4.3/
simsimd-5.4.3/LICENSE
simsimd-5.4.3/PKG-INFO
simsimd-5.4.3/README.md
simsimd-5.4.3/c/
simsimd-5.4.3/c/lib.c
simsimd-5.4.3/pyproject.toml
simsimd-5.4.3/python/
simsimd-5.4.3/python/annotations/
simsimd-5.4.3/python/annotations/__init__.pyi
simsimd-5.4.3/python/annotations/py.typed
simsimd-5.4.3/python/lib.c
simsimd-5.4.3/setup.cfg
simsimd-5.4.3/setup.py
simsimd-5.4.3/simsimd.egg-info/
simsimd-5.4.3/simsimd.egg-info/PKG-INFO
simsimd-5.4.3/simsimd.egg-info/SOURCES.txt
simsimd-5.4.3/simsimd.egg-info/dependency_links.txt
simsimd-5.4.3/simsimd.egg-info/not-zip-safe
simsimd-5.4.3/simsimd.egg-info/top_level.txt

Expected behavior

Expected the simsimd package to be installed.

SimSIMD version

v5.4.3

Operating System

Debian GNU/Linux 12 (bookworm)

Hardware architecture

x86

Which interface are you using?

Python bindings

Contact Details

jimthompson5802@gmail.com

Are you open to being tagged as a contributor?

Is there an existing issue for this?

Code of Conduct

ashvardanian commented 1 month ago

Thanks for spotting, @jimthompson5802! I believe I’ve had a similar problem in USearch and may have fixed it by supplying include_dirs=… in setup.py. I’d appreciate if you could open a PR. Otherwise, I can look at it after Friday.

jimthompson5802 commented 1 month ago

@ashvardanian I'll look to create a PR with your suggested fix.

thank you for the quick response.

jimthompson5802 commented 1 month ago

@ashvardanian Took a stab at your suggestion of include_dirs=... from USearch. Did not seem to work. Following are the details.

I'm assuming you are referencing this section of USearch setup.py:

include_dirs = [
    "include",
    "python",
    "stringzilla/include",
]

and here

setup(
    name=__lib_name__,
    version=__version__,
    packages=["usearch"],
    package_dir={"usearch": "python/usearch"},
    description="Smaller & Faster Single-File Vector Search Engine from Unum",
    author="Ash Vardanian",
    author_email="info@unum.cloud",
    url="https://github.com/unum-cloud/usearch",
    long_description=long_description,
    long_description_content_type="text/markdown",
    license="Apache-2.0",
    classifiers=[
        "Development Status :: 5 - Production/Stable",
        "Natural Language :: English",
        "Intended Audience :: Developers",
        "Intended Audience :: Information Technology",
        "License :: OSI Approved :: Apache Software License",
        "Programming Language :: C++",
        "Programming Language :: Python :: 3 :: Only",
        "Programming Language :: Python :: Implementation :: CPython",
        "Programming Language :: Java",
        "Programming Language :: JavaScript",
        "Programming Language :: Objective C",
        "Programming Language :: Rust",
        "Programming Language :: Other",
        "Operating System :: MacOS",
        "Operating System :: Unix",
        "Operating System :: Microsoft :: Windows",
        "Topic :: System :: Clustering",
        "Topic :: Database :: Database Engines/Servers",
        "Topic :: Scientific/Engineering :: Artificial Intelligence",
    ],
    include_dirs=include_dirs,

Assuming the above is correct, I tried same construct in simsimd setup.py as follows:

I then tried building the source distribution file with this command:

python -m build --sdist

Unfortunately, the resulting simsimd-5.4.3.tar.gz does not contain the include sub-directory contents. It contains this...

/workspace# tar -tzf dist/simsimd-5.4.3.tar.gz 
simsimd-5.4.3/
simsimd-5.4.3/LICENSE
simsimd-5.4.3/PKG-INFO
simsimd-5.4.3/README.md
simsimd-5.4.3/c/
simsimd-5.4.3/c/lib.c
simsimd-5.4.3/pyproject.toml
simsimd-5.4.3/python/
simsimd-5.4.3/python/annotations/
simsimd-5.4.3/python/annotations/__init__.pyi
simsimd-5.4.3/python/annotations/py.typed
simsimd-5.4.3/python/lib.c
simsimd-5.4.3/setup.cfg
simsimd-5.4.3/setup.py
simsimd-5.4.3/simsimd.egg-info/
simsimd-5.4.3/simsimd.egg-info/PKG-INFO
simsimd-5.4.3/simsimd.egg-info/SOURCES.txt
simsimd-5.4.3/simsimd.egg-info/dependency_links.txt
simsimd-5.4.3/simsimd.egg-info/not-zip-safe
simsimd-5.4.3/simsimd.egg-info/top_level.txt

I'll continue working on this to see if I can arrive at the correct setup configuration. I am open to suggestions.

ashvardanian commented 1 month ago

Gotcha! Either way, feel free to open a PR at any point, I can progress from there 🤗