ValueError: signal only works in main thread of the main interpreter

aortiz-WW commented 1 year ago

System Info

My requirements.txt is this: accelerate==0.21.0 aiohttp==3.8.5 aiosignal==1.3.1 altair==5.0.1 anyio==3.7.1 async-timeout==4.0.3 attrs==23.1.0 backoff==2.2.1 beautifulsoup4==4.12.2 bitsandbytes==0.41.1 blinker==1.6.2 bs4==0.0.1 cachetools==5.3.1 certifi==2023.7.22 cffi==1.15.1 charset-normalizer==3.2.0 chroma-hnswlib==0.7.2 chromadb==0.4.6 click==8.1.7 cmake==3.27.2 coloredlogs==15.0.1 cryptography==41.0.3 dataclasses-json==0.5.14 einops==0.6.1 exceptiongroup==1.1.3 fastapi==0.99.1 filelock==3.12.2 flatbuffers==23.5.26 frozenlist==1.4.0 fsspec==2023.6.0 gitdb==4.0.10 GitPython==3.1.32 greenlet==2.0.2 h11==0.14.0 httptools==0.6.0 huggingface-hub==0.16.4 humanfriendly==10.0 idna==3.4 importlib-metadata==6.8.0 importlib-resources==6.0.1 Jinja2==3.1.2 joblib==1.3.2 jsonschema==4.19.0 jsonschema-specifications==2023.7.1 langchain==0.0.267 langsmith==0.0.24 lit==16.0.6 markdown-it-py==3.0.0 MarkupSafe==2.1.3 marshmallow==3.20.1 mdurl==0.1.2 monotonic==1.6 mpmath==1.3.0 multidict==6.0.4 mypy-extensions==1.0.0 networkx==3.1 nltk==3.8.1 numexpr==2.8.5 numpy==1.25.2 nvidia-cublas-cu11==11.10.3.66 nvidia-cuda-cupti-cu11==11.7.101 nvidia-cuda-nvrtc-cu11==11.7.99 nvidia-cuda-runtime-cu11==11.7.99 nvidia-cudnn-cu11==8.5.0.96 nvidia-cufft-cu11==10.9.0.58 nvidia-curand-cu11==10.2.10.91 nvidia-cusolver-cu11==11.4.0.1 nvidia-cusparse-cu11==11.7.4.91 nvidia-nccl-cu11==2.14.3 nvidia-nvtx-cu11==11.7.91 onnxruntime==1.15.1 openapi-schema-pydantic==1.2.4 overrides==7.4.0 packaging==23.1 pandas==2.0.3 pdfminer.six==20221105 Pillow==9.5.0 posthog==3.0.2 protobuf==4.24.0 psutil==5.9.5 pulsar-client==3.2.0 pyarrow==12.0.1 pycparser==2.21 pydantic==1.10.12 pydeck==0.8.0 Pygments==2.16.1 Pympler==1.0.1 PyPika==0.48.9 python-dateutil==2.8.2 python-dotenv==1.0.0 pytz==2023.3 pytz-deprecation-shim==0.1.0.post0 PyYAML==6.0.1 referencing==0.30.2 regex==2023.8.8 requests==2.31.0 rich==13.5.2 rpds-py==0.9.2 safetensors==0.3.2 scikit-learn==1.3.0 scipy==1.11.2 sentence-transformers==2.2.2 sentencepiece==0.1.99 six==1.16.0 smmap==5.0.0 sniffio==1.3.0 soupsieve==2.4.1 SQLAlchemy==2.0.20 starlette==0.27.0 streamlit==1.25.0 sympy==1.12 tenacity==8.2.3 threadpoolctl==3.2.0 tokenizers==0.13.3 toml==0.10.2 toolz==0.12.0 torch==2.0.1 torchvision==0.15.2 tornado==6.3.3 tqdm==4.66.1 transformers==4.31.0 triton==2.0.0 typing-inspect==0.9.0 typing_extensions==4.7.1 tzdata==2023.3 tzlocal==4.3.1 urllib3==2.0.4 uvicorn==0.23.2 uvloop==0.17.0 validators==0.21.2 watchdog==3.0.0 watchfiles==0.19.0 websockets==11.0.3 yarl==1.9.2 zipp==3.16.2

I am following this video tutorial: https://www.youtube.com/watch?v=rIV1EseKwU4 In the app.py I have this following code:

import streamlit as st
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
from transformers import pipeline
import torch
import base64
import textwrap
from langchain.embeddings import HuggingFaceInstructEmbeddings
from langchain.vectorstores import Chroma 
from langchain.chains import RetrievalQA
from langchain.llms import HuggingFacePipeline
from constants import CHROMA_SETTINGS

checkpoint = "falcon-40b/"
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModelForSeq2SeqLM.from_pretrained(
    checkpoint,
    device_map='auto',
    torch_dtype=torch.float32
)

@st.cache_resource
def llm_pipeline():
    pipe = pipeline(
        "text2text-generation",
        model = model,
        tokenizer=tokenizer,
        max_length = 1024,
        do_sample = True,
        temperature = 0.1,
        top_p = 0.95
    )

    local_llm = HuggingFacePipeline(pipeline = pipe)

    return local_llm

@st.cache_resource
def qa_llm():
    llm = llm_pipeline()
    embeddings = HuggingFaceInstructEmbeddings(model_name="all-MiniLM-L6-v2")
    db = Chroma(persist_directory="db", embedding_function=embeddings, client_settings=CHROMA_SETTINGS)
    retriever = db.as_retriever()
    qa = RetrievalQA.from_chain_type(
        llm = llm,
        chain_type='stuff',
        retriever=retriever,
        return_source_document = True
    )

    return qa

def process_answer(instruction):
    response = ""
    instruction = instruction

    qa = qa_llm()
    generate_text = qa(instruction)
    answer = generate_text('result')

    return answer, generate_text

def main():
    st.title('Search Your PDF')
    with st.expander('About App'):
        st.markdown(
            """
            This a generative AI powered question answering app that responds to question about your PDFs
            """
        )

    question = st.text_area('Enter Your Question')
    if st.button('Search'):
        st.info('Your question: ' + question)
        st.info('Your Answer: ')
        answer, metadata = process_answer(question)
        st.write(answer)
        st.write(metadata)

if __name__=="__main__":
    main()

The error I am receiving is this:

ValueError: signal only works in main thread of the main interpreter
Traceback:
File "/home/aortiz/chat-PDF/.venv/lib/python3.10/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 552, in _run_script
    exec(code, module.__dict__)
File "/home/aortiz/chat-PDF/app.py", line 15, in <module>
    model = AutoModelForSeq2SeqLM.from_pretrained(
File "/home/aortiz/chat-PDF/.venv/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 461, in from_pretrained
    config, kwargs = AutoConfig.from_pretrained(
File "/home/aortiz/chat-PDF/.venv/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 986, in from_pretrained
    trust_remote_code = resolve_trust_remote_code(
File "/home/aortiz/chat-PDF/.venv/lib/python3.10/site-packages/transformers/dynamic_module_utils.py", line 535, in resolve_trust_remote_code
    signal.signal(signal.SIGALRM, _raise_timeout_error)
File "/usr/lib/python3.10/signal.py", line 56, in signal
    handler = _signal.signal(_enum_to_int(signalnum), _enum_to_int(handler))

Who can help?

@ArthurZucker , @younesbelkada , @sgugger , @stevhliu , @MKhalusova

Information

[ ] The official example scripts
[X] My own modified scripts

Tasks

[ ] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
[X] My own task or dataset (give details below)

Reproduction

Follow this video instructions to reproduce the problem: https://www.youtube.com/watch?v=rIV1EseKwU4

Expected behavior

It is supposed to be running smoothly on streamlit and I should be able to open the page on the webbrowser

sgugger commented 1 year ago

While the error is not informative (should be fixed in the PR mentioned above) this is because you are using a model using code not in the Transformers library so you need to set trust_remote_code=True in your call to from_pretrained after making sure the code in the folder falcon does not contain anything malicious.

github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

huggingface / transformers