-
### Describe the bug
Cannot use openllm locally at all due to CalledProcessError
### To reproduce
from langchain.llms import OpenLLM
llm = OpenLLM(model_name='falcon', model_id='tiiuae/falcon-40b-…
-
2023-07-20 17:47:04.335 | DEBUG | tools.doc_qa:get_documents:82 - Loading documents...…
-
### Describe the bug
Trying to run a LLaMa 13B model, I encounter a Segmentation Fault error
### To reproduce
Installed OpenLLM by running
```bash
pip install "openllm[llama, vllm, fine-tune]"
p…
-
### Describe the bug
I am trying to deploy GPTQ Llama-2 model using OpenLLM. Did not use any configurations, but simply ran the line from the readme documentation.
When tracing the error, it seems…
-
### Describe the bug
Tried to follow the basic example of the client in the readme.
### To reproduce
```bash
conda create -n openllm-test
conda activate openllm-test
pip install openllm==0.2.20
…
-
Hi,
I'm Mr. Martian, a very friendly guy.
These two commands give me connection refused error on every machine I have tested:
export OPENLLM_ENDPOINT=http://localhost:3000
openllm query 'Exp…
-
Hi there,
I want to fine-tune the openly 13b via the adapter.py and I want to evaluate the performance using two (or more) GPU's before buying.
Currently I tried multiple instances via vast.ai (whic…
-
### Describe the bug
Hi,
I'm trying to run inference against llama2, more precisely "meta-llama/Llama-2-7b-chat-hf".
I installed torch 2.0.1+cu118 and cuda 11.8.
I followed instruction in th…
-
### Describe the bug
As recommended as a stopgap measure in [issue 299](https://github.com/bentoml/OpenLLM/issues/299), I installed OpenLLM with the `--no-binary` flag and tried to launch and query…
-
### Feature request
I would like it if there was a way to convert Llama models to float16 from the openllm CLI. This might look like the following:
```
openllm start llama --model-id meta-llama/L…