-
### Describe the bug
When trying to use Flan-T5 model I keep getting the following error:
```
ValueError: You have to specify either decoder_input_ids or decoder_inputs_embeds
```
I follow the …
-
I see this line in the README: Visit http://localhost:3000/docs.json for OpenLLM's API specification.
Can we get the documentation without having to install/run openllm? I don't have a GPU in my lo…
-
### Feature request
It would be nice to have the option to use AMD GPUs that support ROCm .
PyTorch seems to support ROCm AMD GPUs on Linux - the following was tested on Ubuntu 22.04.2 LTS with …
-
### Describe the bug
After installating openllm, we run
`openllm models list`
### To reproduce
`pip install openllm`
`openllm models`
### Environment
bentoml==1.1.6
openllm==0.3.6…
-
### Bug Description
I have noticed that parameters titles are always ParameterName. In openai json we can fill the parameter description, pydantic is supporting title for parameter, but I don't see…
snarb updated
7 months ago
-
### Describe the bug
In Mac OS, starcoder does not even load, probably because it has no Nvidia GPU.
In Windows, the main issue is the dependency on the `bitsandbytes` library. Since the makers …
-
Do you guys test your model on classical benchmarks such as MMLU, GSM8K, HumanEVAL?
Perhaps the relative improvement on AlpacaEval is merely due to GPT-4's preference for longer responses.
-
### Feature request
how to change port number from default 3000
### Motivation
i need to solve the port confilcting in my system.
### Other
_No response_
-
Hello everyone! I found Llama models like `beomi/llama-2-ko-7b` are giving junk output like `\n[/INST]\n\n[/INST]...`. I tried with multiple Llama2 korean models and I am getting similar junk results.…
-
### Describe the bug
When we serve models like `"openllm start falcon --model-id tiiuae/falcon-7b-instruct"` interaction with this model works fine
via cmd tool
```
export OPENLLM_ENDPOINT=http…