kyamagu / faiss-wheels

Unofficial faiss wheel builder
MIT License
275 stars 35 forks source link

Not Installing #96

Closed Dawood-khan-007 closed 4 months ago

Dawood-khan-007 commented 5 months ago

Describe the bug A clear and concise description of what the bug is.

To Reproduce Describe the steps to reproduce the behavior:Collecting faiss-cpu Using cached faiss-cpu-1.7.4.tar.gz (57 kB) Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Building wheels for collected packages: faiss-cpu Building wheel for faiss-cpu (pyproject.toml) ... error error: subprocess-exited-with-error

× Building wheel for faiss-cpu (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [8 lines of output] running bdist_wheel running build running build_py running build_ext building 'faiss._swigfaiss' extension swigging faiss\faiss\python\swigfaiss.i to faiss\faiss\python\swigfaiss_wrap.cpp swig.exe -python -c++ -Doverride= -I/usr/local/include -Ifaiss -doxygen -DSWIGWIN -module swigfaiss -o faiss\faiss\python\swigfaiss_wrap.cpp faiss\faiss\python\swigfaiss.i error: command 'swig.exe' failed: None [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for faiss-cpu Failed to build faiss-cpu ERROR: Could not build wheels for faiss-cpu, which is required to install pyproject.toml-based projects

pip install faiss-cpu

Expected behavior A clear and concise description of what you expected to happen.

Desktop (please complete the following information):

Additional context Add any other context about the problem here.

kyamagu commented 5 months ago

Specify the environment

dhwanil-thakkar commented 5 months ago

Does the same for me as well, I am working in a Miniconda environment.

msdrigg commented 5 months ago

I am having the same problem with all versions of faiss-cpu:

Here are all the relevant logs

python -V
Python 3.12.1
python -m pip -V
pip 24.0 from /path/lib/python3.12/site-packages/pip (python 3.12)
/usr/bin/swig -version

SWIG Version 4.0.2

Compiled with g++ [x86_64-pc-linux-gnu]

Configured options: +pcre

Please see http://www.swig.org for reporting bugs and further information
pip wheel --no-cache-dir --use-pep517 "faiss-cpu (==1.7.4)"
Collecting faiss-cpu==1.7.4
  Downloading faiss-cpu-1.7.4.tar.gz (57 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.4/57.4 kB 535.6 kB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: faiss-cpu
  Building wheel for faiss-cpu (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for faiss-cpu (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [93 lines of output]
      running bdist_wheel
      running build
      running build_py
      running build_ext
      building 'faiss._swigfaiss' extension
      swigging faiss/faiss/python/swigfaiss.i to faiss/faiss/python/swigfaiss_wrap.cpp
      swig -python -c++ -Doverride= -I/usr/local/include -Ifaiss -doxygen -DSWIGWORDSIZE64 -module swigfaiss -o faiss/faiss/python/swigfaiss_wrap.cpp faiss/faiss/python/swigfaiss.i
      faiss/faiss/python/swigfaiss.i:268: Error: Unable to find 'faiss/impl/platform_macros.h'
      faiss/faiss/python/swigfaiss.i:272: Error: Unable to find 'faiss/utils/ordered_key_value.h'
      faiss/faiss/python/swigfaiss.i:273: Error: Unable to find 'faiss/utils/Heap.h'
      faiss/faiss/python/swigfaiss.i:278: Error: Unable to find 'faiss/utils/AlignedTable.h'
      faiss/faiss/python/swigfaiss.i:279: Error: Unable to find 'faiss/utils/partitioning.h'
      faiss/faiss/python/swigfaiss.i:280: Error: Unable to find 'faiss/utils/hamming.h'
      faiss/faiss/python/swigfaiss.i:281: Error: Unable to find 'faiss/utils/hamming_distance/common.h'
      faiss/faiss/python/swigfaiss.i:386: Error: Unable to find 'faiss/utils/utils.h'
      faiss/faiss/python/swigfaiss.i:387: Error: Unable to find 'faiss/utils/distances.h'
      faiss/faiss/python/swigfaiss.i:388: Error: Unable to find 'faiss/utils/random.h'
      faiss/faiss/python/swigfaiss.i:389: Error: Unable to find 'faiss/utils/sorting.h'
      faiss/faiss/python/swigfaiss.i:391: Error: Unable to find 'faiss/MetricType.h'
      faiss/faiss/python/swigfaiss.i:396: Error: Unable to find 'faiss/Index.h'
      faiss/faiss/python/swigfaiss.i:398: Error: Unable to find 'faiss/impl/DistanceComputer.h'
      faiss/faiss/python/swigfaiss.i:401: Error: Unable to find 'faiss/IndexFlatCodes.h'
      faiss/faiss/python/swigfaiss.i:402: Error: Unable to find 'faiss/IndexFlat.h'
      faiss/faiss/python/swigfaiss.i:403: Error: Unable to find 'faiss/Clustering.h'
      faiss/faiss/python/swigfaiss.i:405: Error: Unable to find 'faiss/utils/extra_distances.h'
      faiss/faiss/python/swigfaiss.i:409: Error: Unable to find 'faiss/impl/Quantizer.h'
      faiss/faiss/python/swigfaiss.i:410: Error: Unable to find 'faiss/impl/ProductQuantizer.h'
      faiss/faiss/python/swigfaiss.i:411: Error: Unable to find 'faiss/impl/AdditiveQuantizer.h'
      faiss/faiss/python/swigfaiss.i:412: Error: Unable to find 'faiss/impl/ResidualQuantizer.h'
      faiss/faiss/python/swigfaiss.i:413: Error: Unable to find 'faiss/impl/LocalSearchQuantizer.h'
      faiss/faiss/python/swigfaiss.i:414: Error: Unable to find 'faiss/impl/ProductAdditiveQuantizer.h'
      faiss/faiss/python/swigfaiss.i:415: Error: Unable to find 'faiss/impl/CodePacker.h'
      faiss/faiss/python/swigfaiss.i:417: Error: Unable to find 'faiss/VectorTransform.h'
      faiss/faiss/python/swigfaiss.i:418: Error: Unable to find 'faiss/IndexPreTransform.h'
      faiss/faiss/python/swigfaiss.i:419: Error: Unable to find 'faiss/IndexRefine.h'
      faiss/faiss/python/swigfaiss.i:420: Error: Unable to find 'faiss/IndexLSH.h'
      faiss/faiss/python/swigfaiss.i:421: Error: Unable to find 'faiss/impl/PolysemousTraining.h'
      faiss/faiss/python/swigfaiss.i:422: Error: Unable to find 'faiss/IndexPQ.h'
      faiss/faiss/python/swigfaiss.i:423: Error: Unable to find 'faiss/IndexAdditiveQuantizer.h'
      faiss/faiss/python/swigfaiss.i:424: Error: Unable to find 'faiss/impl/io.h'
      faiss/faiss/python/swigfaiss.i:426: Error: Unable to find 'faiss/invlists/InvertedLists.h'
      faiss/faiss/python/swigfaiss.i:427: Error: Unable to find 'faiss/invlists/InvertedListsIOHook.h'
      faiss/faiss/python/swigfaiss.i:429: Error: Unable to find 'faiss/invlists/BlockInvertedLists.h'
      faiss/faiss/python/swigfaiss.i:430: Error: Unable to find 'faiss/invlists/DirectMap.h'
      faiss/faiss/python/swigfaiss.i:431: Error: Unable to find 'faiss/IndexIVF.h'
      faiss/faiss/python/swigfaiss.i:436: Error: Unable to find 'faiss/IVFlib.h'
      faiss/faiss/python/swigfaiss.i:437: Error: Unable to find 'faiss/impl/ScalarQuantizer.h'
      faiss/faiss/python/swigfaiss.i:438: Error: Unable to find 'faiss/IndexScalarQuantizer.h'
      faiss/faiss/python/swigfaiss.i:439: Error: Unable to find 'faiss/IndexIVFSpectralHash.h'
      faiss/faiss/python/swigfaiss.i:440: Error: Unable to find 'faiss/IndexIVFAdditiveQuantizer.h'
      faiss/faiss/python/swigfaiss.i:441: Error: Unable to find 'faiss/impl/HNSW.h'
      faiss/faiss/python/swigfaiss.i:442: Error: Unable to find 'faiss/IndexHNSW.h'
      faiss/faiss/python/swigfaiss.i:444: Error: Unable to find 'faiss/impl/kmeans1d.h'
      faiss/faiss/python/swigfaiss.i:447: Error: Unable to find 'faiss/impl/NNDescent.h'
      faiss/faiss/python/swigfaiss.i:448: Error: Unable to find 'faiss/IndexNNDescent.h'
      faiss/faiss/python/swigfaiss.i:450: Error: Unable to find 'faiss/IndexIVFFlat.h'
      faiss/faiss/python/swigfaiss.i:451: Error: Unable to find 'faiss/impl/NSG.h'
      faiss/faiss/python/swigfaiss.i:452: Error: Unable to find 'faiss/IndexNSG.h'
      faiss/faiss/python/swigfaiss.i:457: Error: Unable to find 'faiss/invlists/OnDiskInvertedLists.h'
      faiss/faiss/python/swigfaiss.i:460: Error: Unable to find 'faiss/impl/lattice_Zn.h'
      faiss/faiss/python/swigfaiss.i:461: Error: Unable to find 'faiss/IndexLattice.h'
      faiss/faiss/python/swigfaiss.i:464: Error: Unable to find 'faiss/IndexIVFPQ.h'
      faiss/faiss/python/swigfaiss.i:465: Error: Unable to find 'faiss/IndexIVFPQR.h'
      faiss/faiss/python/swigfaiss.i:466: Error: Unable to find 'faiss/Index2Layer.h'
      faiss/faiss/python/swigfaiss.i:468: Error: Unable to find 'faiss/IndexFastScan.h'
      faiss/faiss/python/swigfaiss.i:469: Error: Unable to find 'faiss/IndexAdditiveQuantizerFastScan.h'
      faiss/faiss/python/swigfaiss.i:470: Error: Unable to find 'faiss/IndexPQFastScan.h'
      faiss/faiss/python/swigfaiss.i:471: Error: Unable to find 'faiss/IndexIVFFastScan.h'
      faiss/faiss/python/swigfaiss.i:472: Error: Unable to find 'faiss/IndexIVFAdditiveQuantizerFastScan.h'
      faiss/faiss/python/swigfaiss.i:473: Error: Unable to find 'faiss/IndexIVFPQFastScan.h'
      faiss/faiss/python/swigfaiss.i:474: Error: Unable to find 'faiss/utils/quantize_lut.h'
      faiss/faiss/python/swigfaiss.i:476: Error: Unable to find 'faiss/IndexBinary.h'
      faiss/faiss/python/swigfaiss.i:477: Error: Unable to find 'faiss/IndexBinaryFlat.h'
      faiss/faiss/python/swigfaiss.i:478: Error: Unable to find 'faiss/IndexBinaryIVF.h'
      faiss/faiss/python/swigfaiss.i:479: Error: Unable to find 'faiss/IndexBinaryFromFloat.h'
      faiss/faiss/python/swigfaiss.i:480: Error: Unable to find 'faiss/IndexBinaryHNSW.h'
      faiss/faiss/python/swigfaiss.i:481: Error: Unable to find 'faiss/IndexBinaryHash.h'
      faiss/faiss/python/swigfaiss.i:483: Error: Unable to find 'faiss/impl/ThreadedIndex.h'
      faiss/faiss/python/swigfaiss.i:487: Error: Unable to find 'faiss/IndexShards.h'
      faiss/faiss/python/swigfaiss.i:490: Error: Unable to find 'faiss/IndexShardsIVF.h'
      faiss/faiss/python/swigfaiss.i:492: Error: Unable to find 'faiss/IndexReplicas.h'
      faiss/faiss/python/swigfaiss.i:496: Error: Unable to find 'faiss/MetaIndexes.h'
      faiss/faiss/python/swigfaiss.i:497: Error: Unable to find 'faiss/IndexIDMap.h'
      faiss/faiss/python/swigfaiss.i:503: Error: Unable to find 'faiss/IndexRowwiseMinMax.h'
      faiss/faiss/python/swigfaiss.i:513: Error: Unable to find 'faiss/impl/AuxIndexStructures.h'
      faiss/faiss/python/swigfaiss.i:514: Error: Unable to find 'faiss/impl/IDSelector.h'
      faiss/faiss/python/swigfaiss.i:516: Error: Unable to find 'faiss/utils/approx_topk/mode.h'
      faiss/faiss/python/swigfaiss.i:753: Error: Unable to find 'faiss/index_io.h'
      faiss/faiss/python/swigfaiss.i:754: Error: Unable to find 'faiss/clone_index.h'
      faiss/faiss/python/swigfaiss.i:758: Error: Unable to find 'faiss/AutoTune.h'
      faiss/faiss/python/swigfaiss.i:759: Error: Unable to find 'faiss/index_factory.h'
      faiss/faiss/python/swigfaiss.i:760: Error: Unable to find 'faiss/MatrixStats.h'
      error: command '/usr/bin/swig' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for faiss-cpu
Failed to build faiss-cpu
ERROR: Failed to build one or more wheels
neofetch
             /////////////                me@pop-os 
         /////////////////////            ---------- 
      ///////*767////////////////         OS: Pop!_OS 22.04 LTS x86_64 
    //////7676767676*//////////////       Host: TRX40 AORUS PRO WIFI -CF 
   /////76767//7676767//////////////      Kernel: 6.6.6-76060606-generic 
  /////767676///*76767///////////////     Uptime: 31 days, 6 hours, 28 mins 
 ///////767676///76767.///7676*///////    Packages: 2334 (dpkg) 
/////////767676//76767///767676////////   Shell: bash 5.1.16 
//////////76767676767////76767/////////   Terminal: node 
///////////76767676//////7676//////////   CPU: AMD Ryzen Threadripper 3960X (48) @ 3.800G 
////////////,7676,///////767///////////   GPU: NVIDIA GeForce GTX 1660 SUPER 
/////////////*7676///////76////////////   Memory: 7672MiB / 31947MiB 
///////////////7676////////////////////
 ///////////////7676///767////////////                            
  //////////////////////'////////////                             
   //////.7676767676767676767,//////
    /////767676767676767676767/////
      ///////////////////////////
         /////////////////////
             /////////////
msdrigg commented 5 months ago

Also getting a similar error on main:

poetry add git+https://github.com/kyamagu/faiss-wheels#main

Updating dependencies
Resolving dependencies... (0.2s)

Package operations: 1 install, 0 updates, 0 removals

  • Installing faiss-cpu (1.7.4 116b034): Failed

  ChefBuildError

  Backend subprocess exited when trying to invoke build_wheel

  running bdist_wheel
  running build
  running build_py
  running build_ext
  building 'faiss._swigfaiss' extension
  swigging faiss/faiss/python/swigfaiss.i to faiss/faiss/python/swigfaiss_wrap.cpp
  swig -python -c++ -Doverride= -doxygen -Ifaiss -I/tmp/tmpl1qimews/.venv/lib/python3.12/site-packages/numpy/core/include -Ifaiss -I/usr/local/include -DSWIGWORDSIZE64 -o faiss/faiss/python/swigfaiss_wrap.cpp faiss/faiss/python/swigfaiss.i
  Unable to find file 'faiss/faiss/python/swigfaiss.i'.
  error: command '/usr/bin/swig' failed with exit code 1

  at ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/installation/chef.py:164 in _prepare
      160│ 
      161│                 error = ChefBuildError("\n\n".join(message_parts))
      162│ 
      163│             if error is not None:
    → 164│                 raise error from None
      165│ 
      166│             return path
      167│ 
      168│     def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:
kyamagu commented 4 months ago

CPython 3.12 is not supported yet following the upstream release.

For those wishing to build a wheel on your own, follow the sdist build step in this workflow: https://github.com/kyamagu/faiss-wheels/blob/main/.github/workflows/build.yml

msdrigg commented 4 months ago

Actually just waiting on a release https://github.com/kyamagu/faiss-wheels/issues/87#issuecomment-1905329587

kiabel commented 4 months ago

I am having the same issue.