search

benman1 / generative_ai_with_langchain

Build large language model (LLM) apps with Python, ChatGPT and other models. This is the companion repository for the book on generative AI with LangChain.
https://amzn.to/43PuIkQ
MIT License
600 stars 242 forks source link

langchain[docarray] doesn't install #31

Closed DerRotMilan closed 7 months ago

DerRotMilan commented 7 months ago

Used requirements.txt to install libs in a fresh docker container. Since I'm planning on using my GPU, I started with nvcr.io/nvidia/rapidsai/notebooks:23.12-cuda12.0-py3.10. All libraries in requirements.txt install with exception of langchain[docarray]. Here I get: `Building wheels for collected packages: hnswlib Building wheel for hnswlib (pyproject.toml) ... error error: subprocess-exited-with-error

× Building wheel for hnswlib (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [55 lines of output] running bdist_wheel running build running build_ext creating tmp gcc -pthread -B /opt/conda/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /opt/conda/include -fPIC -O2 -isystem /opt/conda/include -fPIC -I/opt/conda/include/python3.10 -c /tmp/tmp2lnz5r_o.cpp -o tmp/tmp2lnz5r_o.o -std=c++14 gcc -pthread -B /opt/conda/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /opt/conda/include -fPIC -O2 -isystem /opt/conda/include -fPIC -I/opt/conda/include/python3.10 -c /tmp/tmpql3olux4.cpp -o tmp/tmpql3olux4.o -std=c++11 Traceback (most recent call last): File "/opt/conda/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in main() File "/opt/conda/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main json_out['return_val'] = hook(hook_input['kwargs']) File "/opt/conda/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel return _build_backend().build_wheel(wheel_directory, config_settings, File "/tmp/pip-build-env-k9nz_xdj/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 410, in build_wheel return self._build_with_temp_dir( File "/tmp/pip-build-env-k9nz_xdj/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 395, in _build_with_temp_dir self.run_setup() File "/tmp/pip-build-env-k9nz_xdj/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 311, in run_setup exec(code, locals()) File "", line 131, in File "/tmp/pip-build-env-k9nz_xdj/overlay/lib/python3.10/site-packages/setuptools/init.py", line 103, in setup return distutils.core.setup(attrs) File "/tmp/pip-build-env-k9nz_xdj/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup return run_commands(dist) File "/tmp/pip-build-env-k9nz_xdj/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands dist.run_commands() File "/tmp/pip-build-env-k9nz_xdj/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands self.run_command(cmd) File "/tmp/pip-build-env-k9nz_xdj/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 963, in run_command super().run_command(command) File "/tmp/pip-build-env-k9nz_xdj/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command cmd_obj.run() File "/tmp/pip-build-env-k9nz_xdj/overlay/lib/python3.10/site-packages/wheel/bdist_wheel.py", line 368, in run self.run_command("build") File "/tmp/pip-build-env-k9nz_xdj/overlay/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command self.distribution.run_command(command) File "/tmp/pip-build-env-k9nz_xdj/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 963, in run_command super().run_command(command) File "/tmp/pip-build-env-k9nz_xdj/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command cmd_obj.run() File "/tmp/pip-build-env-k9nz_xdj/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/build.py", line 131, in run self.run_command(cmd_name) File "/tmp/pip-build-env-k9nz_xdj/overlay/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command self.distribution.run_command(command) File "/tmp/pip-build-env-k9nz_xdj/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 963, in run_command super().run_command(command) File "/tmp/pip-build-env-k9nz_xdj/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command cmd_obj.run() File "/tmp/pip-build-env-k9nz_xdj/overlay/lib/python3.10/site-packages/setuptools/command/build_ext.py", line 89, in run _build_ext.run(self) File "/tmp/pip-build-env-k9nz_xdj/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run self.build_extensions() File "", line 101, in build_extensions File "", line 70, in cpp_flag RuntimeError: Unsupported compiler -- at least C++11 support is needed! [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for hnswlib Failed to build hnswlib ERROR: Could not build wheels for hnswlib, which is required to install pyproject.toml-based projects` Looks like it's failing due to an unsupported C++ compiler. Will have a closer look.

DerRotMilan commented 7 months ago

As suggested, I checked the version of langchain and langsmith:

**langchain                                0.1.10**
langchain-community                      0.0.25
langchain-core                           0.1.28
langchain-decorators                     0.5.4
langchain-experimental                   0.0.53
langchain-text-splitters                 0.0.1
langdetect                               1.0.9
**langsmith                                0.1.14**

Not the same versions as in requirements.txt - supposedly because of the error above. So I installed the C++ compiler (via apt-get install build-essential -y but then get the following error: ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. langchain-community 0.0.25 requires langsmith<0.2.0,>=0.1.0, but you have langsmith 0.0.92 which is incompatible. langchain-core 0.1.28 requires langsmith<0.2.0,>=0.1.0, but you have langsmith 0.0.92 which is incompatible. langchain-experimental 0.0.53 requires langchain<0.2.0,>=0.1.8, but you have langchain 0.0.284 which is incompatible. pyppeteer 1.0.2 requires urllib3<2.0.0,>=1.25.8, but you have urllib3 2.2.1 which is incompatible. unstructured-client 0.21.0 requires dataclasses-json>=0.6.4, but you have dataclasses-json 0.5.14 which is incompatible.

benman1 commented 7 months ago

Hi @DerRotMilan That sounds cool what you're up to!

You seem to have the wrong compiler.

I think you already nearly found the solution yourself. I was going to suggest adding installs of build-essential and python3-dev, something like this to your Dockerfile:

apt install -y build-essential python3-dev

python3-dev might not be needed.

Since the environment is already messed up, I'd suggest starting from scratch once you've added the line. These versions don't match the repo as you correctly observed.