search

su77ungr / CASALIOY

♾️ toolkit for air-gapped LLMs on consumer-grade hardware
Apache License 2.0
229 stars 31 forks source link

Streamlit requirements cause install error #30

Closed hippalectryon-0 closed 1 year ago

hippalectryon-0 commented 1 year ago

Tested both on windows 10 & ubuntu 22.

Problem: python -m pip install -r requirements.txt fails with the latest addition of streamlit==1.22.0. This seems to be due to a requirement grpcio-tools (full log here):

...
x86_64-linux-gnu-gcc -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DHAVE_PTHREAD=1 -I. -Igrpc_root -Igrpc_root/include -Ithird_party/protobuf/src -I/home/ab263315/PycharmProjects/CASALIOY/venv/include -I/usr/include/python3.11 -c third_party/protobuf/src/google/protobuf/wrappers.pb.cc -o build/temp.linux-x86_64-cpython-311/third_party/protobuf/src/google/protobuf/wrappers.pb.o -std=c++14 -fno-wrapv -frtti
      x86_64-linux-gnu-gcc -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DHAVE_PTHREAD=1 -I. -Igrpc_root -Igrpc_root/include -Ithird_party/protobuf/src -I/home/ab263315/PycharmProjects/CASALIOY/venv/include -I/usr/include/python3.11 -c third_party/protobuf/src/google/protobuf/util/type_resolver_util.cc -o build/temp.linux-x86_64-cpython-311/third_party/protobuf/src/google/protobuf/util/type_resolver_util.o -std=c++14 -fno-wrapv -frtti
      x86_64-linux-gnu-gcc -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DHAVE_PTHREAD=1 -I. -Igrpc_root -Igrpc_root/include -Ithird_party/protobuf/src -I/home/ab263315/PycharmProjects/CASALIOY/venv/include -I/usr/include/python3.11 -c third_party/protobuf/src/google/protobuf/descriptor.pb.cc -o build/temp.linux-x86_64-cpython-311/third_party/protobuf/src/google/protobuf/descriptor.pb.o -std=c++14 -fno-wrapv -frtti
      error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for grpcio-tools
  Running setup.py clean for grpcio-tools
  Building wheel for st-annotated-text (setup.py) ... done
  Created wheel for st-annotated-text: filename=st_annotated_text-4.0.0-py3-none-any.whl size=8904 sha256=729499689c74c921c118f9cf6e38f66926bf24f8b7d454f583df4170ad9c69e5
  Stored in directory: /home/ab263315/.cache/pip/wheels/6b/6a/df/1eda8d742a9094f5694398f5a81a4eb8297297b2cf9f027342
  Building wheel for validators (setup.py) ... done
  Created wheel for validators: filename=validators-0.20.0-py3-none-any.whl size=19579 sha256=5a11acee4f5c3af0a3af713106f43689392b308f4ca499e839a21c203ce7e488
  Stored in directory: /home/ab263315/.cache/pip/wheels/82/35/dc/f88ec71edf2a5596bd72a8fa1b697277e0fcd3cde83048b8bf
  Building wheel for python-docx (setup.py) ... done
  Created wheel for python-docx: filename=python_docx-0.8.11-py3-none-any.whl size=184491 sha256=7e6078c24e43320edef649b0a75cda6bd85085d33c1b1b1411262bf734998660
  Stored in directory: /home/ab263315/.cache/pip/wheels/b2/11/b8/209e41af524253c9ba6c2a8b8ecec0f98ecbc28c732512803c
  Building wheel for python-pptx (setup.py) ... done
  Created wheel for python-pptx: filename=python_pptx-0.6.21-py3-none-any.whl size=470935 sha256=ece0c1b144342dac31b878d220cbc348195453e3b77a3f752756613af34a3266
  Stored in directory: /home/ab263315/.cache/pip/wheels/f4/c7/af/d1d91f3decfaa7621033f30b69a29bf0b1206005663d233e7a
  Building wheel for olefile (setup.py) ... done
  Created wheel for olefile: filename=olefile-0.46-py2.py3-none-any.whl size=35417 sha256=7ae136ecdc319f13e6f9bfe34fd0bb898cceea5e149a46720044b69504595f55
  Stored in directory: /home/ab263315/.cache/pip/wheels/7a/28/c9/4745d0108b03ae5933fd107bd3946eec0d9fa794f8ce837a46
Successfully built pygpt4all llama-cpp-python pandoc htbuilder st-annotated-text validators python-docx python-pptx olefile
Failed to build grpcio-tools
ERROR: Could not build wheels for grpcio-tools, which is required to install pyproject.toml-based projects

Edit: if you wonder how I got the GUI working before: I hadn't installed the "new" requirements, and pip somehow managed to install conflicting versions of the packages (ex protobuf>=4) that got the whole thing working, but that's an anomaly

alxspiker commented 1 year ago

Can you try:

pip install --upgrade setuptools
hippalectryon-0 commented 1 year ago

pip install --upgrade setuptools

Already done, no change

Update: the issue comes when streamlit==1.22.0 and qdrant-client==1.1.7 are together. If we remove either, the install works fine. I believe this is because streamlit forces protobuf<4, and qdrant-client then installs some obscure version of grpcio-tools.

hippalectryon-0 commented 1 year ago

Related: https://github.com/streamlit/streamlit/issues/5626

According to this, this is indeed what is going on, and the next version of streamlit should fix that.

I use python3.11, which is probably why you don't have this error on your end.

alxspiker commented 1 year ago

Got this error myself:

Traceback (most recent call last):
  File "C:\Users\Haley The Retard\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\Haley The Retard\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Users\Haley The Retard\AppData\Local\Programs\Python\Python310\Scripts\streamlit.exe\__main__.py", line 4, in <module>
  File "C:\Users\Haley The Retard\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\__init__.py", line 55, in <module>
    from streamlit.delta_generator import DeltaGenerator as _DeltaGenerator
  File "C:\Users\Haley The Retard\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\delta_generator.py", line 36, in <module>     
    from streamlit import config, cursor, env_util, logger, runtime, type_util, util
  File "C:\Users\Haley The Retard\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\cursor.py", line 18, in <module>
    from streamlit.runtime.scriptrunner import get_script_run_ctx
  File "C:\Users\Haley The Retard\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\runtime\__init__.py", line 16, in <module>    
    from streamlit.runtime.runtime import Runtime as Runtime
  File "C:\Users\Haley The Retard\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\runtime\runtime.py", line 29, in <module>     
    from streamlit.proto.BackMsg_pb2 import BackMsg
  File "C:\Users\Haley The Retard\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\proto\BackMsg_pb2.py", line 14, in <module>   
    from streamlit.proto import ClientState_pb2 as streamlit_dot_proto_dot_ClientState__pb2
  File "C:\Users\Haley The Retard\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\proto\ClientState_pb2.py", line 14, in <module>
    from streamlit.proto import WidgetStates_pb2 as streamlit_dot_proto_dot_WidgetStates__pb2
  File "C:\Users\Haley The Retard\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\proto\WidgetStates_pb2.py", line 14, in <module>
    from streamlit.proto import Common_pb2 as streamlit_dot_proto_dot_Common__pb2
  File "C:\Users\Haley The Retard\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\proto\Common_pb2.py", line 36, in <module>    
    _descriptor.FieldDescriptor(
  File "C:\Users\Haley The Retard\AppData\Local\Programs\Python\Python310\lib\site-packages\google\protobuf\descriptor.py", line 561, in __new__    
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

To fix (Worked for me): pip install --force protobuf==3.20.3

Possible fix: #31

hippalectryon-0 commented 1 year ago

I've ended up installing streamlit from the rep while we wait for 1.23.x. I'll sumbit a PR soon (I'm using poetry)

hippalectryon-0 commented 1 year ago

32