c0sogi llama-api issues

c0sogi / llama-api

An OpenAI-like LLaMA inference API

MIT License

110 stars 9 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

FastAPI + llamapi issue

#29 Samraw003 opened 3 months ago
0
Stopped working after enabling CUDA

#28 alexellis opened 7 months ago
0
High RAM and CPU usage

#27 delta-whiplash opened 8 months ago
0
Usage of embedding through langchain

#26 jordandroid opened 8 months ago
0
Support min_p sampler

#25 atisharma opened 9 months ago
0
How can I use a specific prompt template?

#24 Dougie777 opened 9 months ago
0
how to run this api in cpu only mode

#23 delta-whiplash opened 9 months ago
1
Zephyr7b gives gobbly gook output but Mistral7b works fine.

#22 Dougie777 opened 10 months ago
0
exllama GPU split

#21 atisharma opened 10 months ago
1
exllamav2

#20 ehartford opened 10 months ago
2
Any way to define embeddings model in model_definitions.py?

#19 morgendigital opened 10 months ago
1
Long generations dont return data but server says 200 OK. Swagger screen just says LOADING forever.

#18 Dougie777 opened 11 months ago
5
BUG: I found the model path bug!

#17 Dougie777 closed 11 months ago
2
Set number of cores being used on cpu?

#16 Dougie777 closed 11 months ago
2
Support for ExLlama V2

#15 Immortalin closed 11 months ago
2
Generation stops at 251 tokens - works fine on oobabooga

#14 Dougie777 closed 11 months ago
3
warning: failed to mlock 245760-byte buffer (after previously locking 0 bytes): Cannot allocate memory llm_load_tensors: mem required = 46494.72 MB (+ 1280.00 MB per state)

#13 Dougie777 closed 11 months ago
4
model_definitions.py

#12 Dougie777 closed 11 months ago
3
Is there a way to use this on google Colab and have the url be public?

#11 ashercn97 opened 11 months ago
1
Dumb question: definitions.py model parameters

#10 Dougie777 closed 11 months ago
2
Proxy to openAI

#9 kreolsky opened 11 months ago
2
Using with LangChain instead openai API

#8 kreolsky opened 11 months ago
1
Dev update (23.9.3.)

#7 c0sogi closed 12 months ago
0
Dev update (23.8.27.)

#6 c0sogi closed 1 year ago
0
Dev update (23.8.22.)

#5 c0sogi closed 1 year ago
0
Dev update (23.8.17.)

#4 c0sogi closed 1 year ago
0
Dev update (23.8.9.)

#3 c0sogi closed 1 year ago
0
Huggingface downloader & Simpler log message & InterruptMixin

#2 c0sogi closed 1 year ago
0
Dependency solution

#1 c0sogi closed 1 year ago
0