Does not work with Python 3.11?

[bortzmeyer]

pip install onprem
...
ERROR: Ignored the following versions that require a different python version: 1.21.2 Requires-Python >=3.7,<3.11; 1.21.3 Requires-Python >=3.7,<3.11; 1.21.4 Requires-Python >=3.7,<3.11; 1.21.5 Requires-Python >=3.7,<3.11; 1.21.6 Requires-Python >=3.7,<3.11
ERROR: Could not find a version that satisfies the requirement onnxruntime>=1.14.1 (from chromadb) (from versions: none)
ERROR: No matching distribution found for onnxruntime>=1.14.1

% python --version
Python 3.11.5

(No problem with a Python 3.10)

[amaiya]

Hello,

I just installed OnPrem.LLM in a mamba Python 3.11 environment and everything worked:

(llm311) amaiya@mctl:~$ ipython
Python 3.11.5 | packaged by conda-forge | (main, Aug 27 2023, 03:34:09) [GCC 12.3.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.15.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from onprem import LLM

In [2]: llm = LLM()

In [3]: saved_output = llm.prompt('What is a cute name for a cat?')
llama.cpp: loading model from /home/amaiya/onprem_data/Wizard-Vicuna-7B-Uncensored.ggmlv3.q4_0.bin
llama_model_load_internal: format     = ggjt v3 (latest)
llama_model_load_internal: n_vocab    = 32000
llama_model_load_internal: n_ctx      = 2048
llama_model_load_internal: n_embd     = 4096
llama_model_load_internal: n_mult     = 256
llama_model_load_internal: n_head     = 32
llama_model_load_internal: n_layer    = 32
llama_model_load_internal: n_rot      = 128
llama_model_load_internal: ftype      = 2 (mostly Q4_0)
llama_model_load_internal: n_ff       = 11008
llama_model_load_internal: model size = 7B
llama_model_load_internal: ggml ctx size =    0.08 MB
llama_model_load_internal: mem required  = 5407.72 MB (+ 1026.00 MB per state)
llama_new_context_with_model: kv self size  = 1024.00 MB

Some examples of cute names for cats include Fluffy, Mittens, Purrsia, and Mr. Bigglesworth.

I'm not sure how your environment is setup, but are you sure your pip command is using Python 3.11 and not Python 2? Perhaps try pip3 install onprem.