amaiya / onprem

A tool for running on-premises large language models with non-public data
https://amaiya.github.io/onprem
Apache License 2.0
692 stars 35 forks source link

Unable to get ingest working #4

Closed smach closed 1 year ago

smach commented 1 year ago

This very well may be my fault, since my Python knowledge is rather basic. But when trying to follow the ingest document examples with my own documents using the code

from onprem import LLM
llm = LLM()
llm.ingest('./sample_docs/')`

I get the error below

RuntimeError: 
    An attempt has been made to start a new process before the
    current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.

This happens both on my Mac and on my Windows PC.

Below code works fine, though.

prompt = """What is a large language model?"""
llm_response = llm.prompt(prompt)
amaiya commented 1 year ago

Hi @smach:

The call to ingest in this Google Colab notebook seems to work fine and all CI tests pass including ones with ingest.

That error usually indicates that you need to place code in a if __name__ == '__main__': guard (as mentioned in this post).

Are you running this in a Jupyter notebook? Or, are you putting the code in a .py file and running it (with python my_file.py, for example)? If the latter, perhaps you may need to do this:

if __name__ == '__main__':    
    from onprem import LLM
    llm = LLM()
    llm.ingest('./sample_docs/')`
smach commented 1 year ago

Thank you, I am indeed running it in a local .py file, and that stops the error. (I'm now having problems with the correct version of pymupdf which probably has nothing to do with this package. I suspect I need to improve my knowledge of Python sysadmin and then try again sometime.)

smach commented 1 year ago

(Closing)