ray-project ray-llm issues

ray-project / ray-llm

RayLLM - LLMs on Ray

https://aviary.anyscale.com

Apache License 2.0

1.2k stars 86 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

Update README.md for RayLLM archival

#152 akshay-anyscale closed 1 month ago
0
[DOC] Add instructions to install and run RayLLM backend locally

#151 xwu99 opened 2 months ago
0
Running ray-llm 0.5.0 on g4dn.12xlarge instance

#150 golemsentience opened 2 months ago
2
Update to latest vLLM upstream and Support vLLM on CPU

#149 xwu99 opened 2 months ago
9
Is this project still actively being maintained?

#148 nkwangleiGIT opened 2 months ago
8
templating serve-config and model config instead of copy and paste

#147 nhaocalgary opened 2 months ago
0
follow up on removing explorer from readme

#146 ArturNiederfahrenhorst closed 3 months ago
0
Remove aviary.anyscale.com references from README

#145 ArturNiederfahrenhorst closed 3 months ago
0
Error when `serve run`

#144 andakai opened 3 months ago
1
RAY-LLM stuck at replica step

#143 NBTrong opened 3 months ago
1
Run rayllm frontend on head pod fails

#142 viirya opened 3 months ago
1
`serve status` fails on the head pod after model is deployed

#141 viirya opened 3 months ago
0
RayWorkerVllm Actor Dies After ~1h: The actor is dead because all references to the actor were removed.

#140 cshyjak opened 4 months ago
0
Issue when running ray-llm with tensorrt-llm

#139 leon-g-xu opened 4 months ago
1
Use vllm 0.3.3

#138 leon-g-xu closed 4 months ago
0
There should be a feature saying that all of the 3 options are wrong

#137 ollie-iterators opened 4 months ago
1
Support vllm 0.3.x

#136 leon-g-xu closed 4 months ago
1
Update deploy-on-eks.md

#135 viirya opened 4 months ago
0
How to adjust engine kwargs from defaults values for models in `./models/`

#134 SamComber closed 4 months ago
0
Autoscaling support in Ray-llm

#133 Jeffwan opened 4 months ago
0
Support for the Mistral based Embeddings models

#132 lynkz-matt-psaltis opened 4 months ago
0
In the 0.5.0 release, some files appear to be missing

#131 lynkz-matt-psaltis opened 5 months ago
0
Serve a new model without restarting RayLLM

#130 k6l3 opened 5 months ago
1
A100 not correctly detected / No available node types can fulfill resource request

#129 wizche opened 5 months ago
0
Add examples to the prompt format docs

#128 alanwguo closed 5 months ago
0
Podman Error on red hat 9?

#127 jayteaftw opened 5 months ago
0
Add more details about prompt format in the docs

#126 alanwguo closed 5 months ago
1
[BUG] workers do not launch on g5.12xlarges for the latest image 0.5.0.

#125 JGSweets opened 5 months ago
6
Error when trying to run tensorrt model on ray

#124 rifkybujana closed 5 months ago
1
Queue-Worker System

#123 AIApprentice101 opened 5 months ago
2
[BUG] Docker -- anyscale/ray-llm:latest is essentially 0.3.0

#122 JGSweets closed 5 months ago
1
[BUG] serve.run host / port are deprecated, Cannot access port 8000 from outside the resource.

#121 JGSweets closed 5 months ago
8
Recently updated to 0.5.0 and can no longer deploy models -- add suitable Node Error

#120 JGSweets closed 5 months ago
2
Update vllm version

#119 sihanwang41 closed 5 months ago
0
Add a example of Azure GPU

#118 chenweisomebody126 opened 5 months ago
1
How to use partial GPU?

#117 rifkybujana opened 5 months ago
2
Anyscale/ray-llm:latest does not contain the serve_configs folder and models/continuous_batching/quantization folders

#116 WinsonSou closed 5 months ago
2
Update doc build ci job

#115 sihanwang41 closed 6 months ago
0
Ray-LLM Head with VLLM Head throws configuration error

#114 lynkz-matt-psaltis opened 6 months ago
2
Loading 13B model across 6 GPUs (Distributed Inference)

#113 WinsonSou closed 6 months ago
2
Permission denied when downloading or serving any model.

#112 Anindyadeep opened 6 months ago
0
Release 0.5.0

#111 sihanwang41 closed 5 months ago
5
Release 0.5.0

#110 sihanwang41 closed 6 months ago
0
Cannot specify models using yaml as a list of dicts or LLMApp objects

#109 victorserbu2709 opened 6 months ago
0
Error: Tokenizer class does not exist when load local model

#108 YixuanCao opened 6 months ago
1
Create Google--GcpGPT

#107 NeelamMandavia opened 6 months ago
1
How to submit a LLM training job?

#106 qingqiuhe opened 6 months ago
2
[New Model] Add an example for Mixtral 8 * 7B Instruct model

#105 jinnig opened 6 months ago
0
Closed

#104 nyagah closed 6 months ago
1
depoyment of quantized models fails

#103 matthiasmfr opened 6 months ago
1