Open JasonWeill opened 6 months ago
# CPU-only version
$ conda install -c pytorch/label/nightly faiss-cpu
# GPU(+CPU) version
$ conda install -c pytorch/label/nightly -c nvidia faiss-gpu=1.7.4
then try to install again
p.s. 2 days ago were merged changes required for successful run on v3.12 python.
p.p.s. for Apple Silicon should be used CPU only version
Install with virtual environment, exec it in project folder, not in your HOME dir
pip install virtualenv
python -m venv myenv
source myenv/bin/activate
pip install jupyter_ai
deactivate
I created a new Conda environment with Python 3.12 on an Apple Silicon Mac, installed JupyterLab 4.0.11, and I tried to run ./scripts/install.sh
from my local Jupyter AI repository's root. Although it fetched the faiss-cpu
1.7.4 wheel, I got an error during the install process. Partial log:
> @jupyter-ai/core:dev-install
@jupyter-ai/core: yarn run v1.22.19
$ pip install -e ".[dev,all,test]" && jupyter labextension develop . --overwrite && jupyter server extension enable jupyter_ai
@jupyter-ai/core: Obtaining file:///Users/jweill/git/jupyter-ai/packages/jupyter-ai
@jupyter-ai/core: Installing build dependencies: started
@jupyter-ai/core: Installing build dependencies: finished with status 'done'
@jupyter-ai/core: Checking if build backend supports build_editable: started
@jupyter-ai/core: Checking if build backend supports build_editable: finished with status 'done'
@jupyter-ai/core: Getting requirements to build editable: started
@jupyter-ai/core: Getting requirements to build editable: finished with status 'done'
@jupyter-ai/core: Installing backend dependencies: started
@jupyter-ai/core: Installing backend dependencies: finished with status 'done'
@jupyter-ai/core: Preparing editable metadata (pyproject.toml): started
@jupyter-ai/core: Preparing editable metadata (pyproject.toml): finished with status 'done'
…
@jupyter-ai/core: Collecting faiss-cpu (from jupyter_ai==2.10.0b0)
@jupyter-ai/core: Downloading faiss-cpu-1.7.4.tar.gz (57 kB)
@jupyter-ai/core: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/57.4 kB ? eta -:--:--
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.4/57.4 kB 6.1 MB/s eta 0:00:00
@jupyter-ai/core: Installing build dependencies: started
@jupyter-ai/core: Installing build dependencies: finished with status 'done'
@jupyter-ai/core: Getting requirements to build wheel: started
@jupyter-ai/core: Getting requirements to build wheel: finished with status 'done'
@jupyter-ai/core: Preparing metadata (pyproject.toml): started
@jupyter-ai/core: Preparing metadata (pyproject.toml): finished with status 'done'
…
@jupyter-ai/core: Building wheels for collected packages: jupyter_ai, faiss-cpu, colored
@jupyter-ai/core: Building editable for jupyter_ai (pyproject.toml): started
@jupyter-ai/core: Building editable for jupyter_ai (pyproject.toml): finished with status 'done'
@jupyter-ai/core: Created wheel for jupyter_ai: filename=jupyter_ai-2.10.0b0-py3-none-any.whl size=6047506 sha256=5dc674fd5c8f69797bf59b8aafda385612ca7acb3ce1410d098d16d23ec65c32
@jupyter-ai/core: Stored in directory: /private/var/folders/md/x69whq_524q4kb6r0gnq3df40000gr/T/pip-ephem-wheel-cache-29jz42ea/wheels/37/f0/07/c80035cdc0507bfc2805314c907280b36cec99fd0028111e67
@jupyter-ai/core: Building wheel for faiss-cpu (pyproject.toml): started
@jupyter-ai/core: Building wheel for faiss-cpu (pyproject.toml): finished with status 'error'
@jupyter-ai/core: error: subprocess-exited-with-error
@jupyter-ai/core:
@jupyter-ai/core: × Building wheel for faiss-cpu (pyproject.toml) did not run successfully.
@jupyter-ai/core: │ exit code: 1
@jupyter-ai/core: ╰─> [8 lines of output]
@jupyter-ai/core: running bdist_wheel
@jupyter-ai/core: running build
@jupyter-ai/core: running build_py
@jupyter-ai/core: running build_ext
@jupyter-ai/core: building 'faiss._swigfaiss' extension
@jupyter-ai/core: swigging faiss/faiss/python/swigfaiss.i to faiss/faiss/python/swigfaiss_wrap.cpp
@jupyter-ai/core: swig -python -c++ -Doverride= -I/usr/local/include -Ifaiss -doxygen -module swigfaiss -o faiss/faiss/python/swigfaiss_wrap.cpp faiss/faiss/python/swigfaiss.i
@jupyter-ai/core: error: command 'swig' failed: No such file or directory
@jupyter-ai/core: [end of output]
@jupyter-ai/core:
@jupyter-ai/core: note: This error originates from a subprocess, and is likely not a problem with pip.
@jupyter-ai/core: ERROR: Failed building wheel for faiss-cpu
@jupyter-ai/core: Building wheel for colored (setup.py): started
@jupyter-ai/core: Building wheel for colored (setup.py): finished with status 'done'
@jupyter-ai/core: Created wheel for colored: filename=colored-1.4.4-py3-none-any.whl size=14249 sha256=f37a7704023a420b0a10c9896470671d2a0aea76e0dd657ba6f55054f7c54463
@jupyter-ai/core: Stored in directory: /Users/jweill/Library/Caches/pip/wheels/7e/5d/d5/b488a46de10f44dcf9d525a2051d51f45474e8dee6ab3da46b
@jupyter-ai/core: Successfully built jupyter_ai colored
@jupyter-ai/core: Failed to build faiss-cpu
@jupyter-ai/core: ERROR: Could not build wheels for faiss-cpu, which is required to install pyproject.toml-based projects
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
——————————————————————————————————————————————————————————————————————————————
> Lerna (powered by Nx) Running target dev-install for 2 projects failed
@OleksandrKucherenko Are you using Apple Silicon locally? Were you able to install using a wheel that works with these processors? https://github.com/kyamagu/faiss-wheels/issues/87 remains open.
:heavy_check_mark: Reproduced on latest Kaggle env, so I switched back in time (2024-01-11) so the following runtime is :ok:
sys.version_info(major=3, minor=10, micro=12, releaselevel='final', serial=0)
https://github.com/kyamagu/faiss-wheels/issues/87 was completed last week, so once its new version is published, we should be able to add Python 3.12 support.
Once the issue closed i'll upgrade the notebook and let you know here :+1:
Progress is being made on this issue.
https://github.com/kyamagu/faiss-wheels/pull/88 was merged, meaning that Python 3.12 wheels will be distributed for future faiss
releases.
faiss
will likely have a release in two weeks: https://github.com/facebookresearch/faiss/issues/3263#issuecomment-1966383696. This means that in two weeks' time, we should be able to bump our faiss
dependency and allow for Python 3.12. 🎉
Link to upstream issue: https://github.com/kyamagu/faiss-wheels/issues/87
Yup, and I also saw the releaseNote of the latest Jupyter: it rocks 😎
If it's still fail, try to install vs_BuildTools.exe first. Actually, you can use jupyter_ai_magic instead of jupyter_ai.
Thanks for feedbacks @JIZUSHANBOJUE , I'm giving it a try :zap:
:hourglass_flowing_sand: Test in progress
It's all :ok_man: I've switcher to latest available Kaggle runtime 3.12, see 🪄 Jupyter-AI magic
:pray: @JIZUSHANBOJUE for the tip
Quick ping - if I'm reading this correctly, faiss 1.8.0 is now out with the required updates. Would it be possible to push up a release here so we can deploy in 3.12 environments that are conda/mamba-managed? No worries if something else is holding things up, but if not, I'd love to have it more widely available :) Thanks team for the great work!
ready to upgrade my test env :sunglasses:
Hi folks, checking in - the conda/mamba install under a 3.12 environment still fails to resolve... If I read the error log correctly, the problem is a pin on tiktoken < 0.4.0, which is incompatible with py3.12...
I tested this in a freshly created conda/mamba environment that only had py3.12 in it, and nothing else, to be sure the incompatibilities weren't being triggered by any other package in my (large) regular env.
Happy to provide further testing if useful, this was all on an intel Mac with conda-forge packages and using mamba as the client.
The dependency on tiktoken was removed, but for me it now chokes on faiss-cpu
environment.yml
channels:
- conda-forge
dependencies:
- python ==3.12 # when removing this pin, it completes fine with python 3.11.9
- jupyter-ai >=2.14.1
Result
$ mamba env create -f environment.yml --name jupyter-ai
conda-forge/osx-64 Using cache
conda-forge/noarch Using cache
pkgs/main/osx-64 No change
pkgs/main/noarch No change
pkgs/r/osx-64 No change
pkgs/r/noarch No change
Looking for: ['python==3.12', "jupyter-ai[version='>=2.14.1']"]
Encountered problems while solving:
- package jupyter-ai-2.14.1-pyhd8ed1ab_0 requires faiss-cpu, but none of the providers can be installed
Yup, can confirm - the full log I get for a similar request is:
The following package could not be installed
└─ jupyter-ai is installable and it requires
└─ faiss-cpu, which requires
└─ faiss [1.7.2 *_cpu|1.7.3 *_cpu|1.7.4 *_cpu] with the potential options
├─ faiss [1.7.2|1.7.3|1.7.4] would require
│ └─ python >=3.10,<3.11.0a0 , which can be installed;
├─ faiss [1.7.2|1.7.3|1.7.4] would require
│ └─ python >=3.11,<3.12.0a0 , which can be installed;
├─ faiss [1.7.2|1.7.3|1.7.4] would require
│ └─ python >=3.8,<3.9.0a0 , which can be installed;
└─ faiss [1.7.2|1.7.3|1.7.4] would require
└─ python >=3.9,<3.10.0a0 , which can be installed.
Please let us know if there's anything we could do to help, thanks for all the great work on this project!
Unfortunately the latest release of faiss-cpu
that supports Python 3.12 is not present on Conda Forge. It seems the feedstock has been unmaintained for some time: https://github.com/conda-forge/faiss-split-feedstock
Furthermore, there doesn't seem to be a way to specify a dependency to be installed only via pip
: https://github.com/conda/conda-build/issues/548
Thanks a lot guys for the followup and giving up :pray:
I would love Python 3.12 support. What's the current status?
Description
Attempting to install Jupyter AI on Python 3.12 is not supported, even though our recipe on
conda-forge
merely stipulatespython >=3.8
.The following error occurs when running
pip install jupyter_ai
with Python 3.12 on macOS 13.6.3 on an Apple Silicon processor:See also https://github.com/kyamagu/faiss-wheels/issues/87 for the lack of Python 3.12 wheels for faiss-cpu.