search

opea-project / GenAIComps

GenAI components at micro-service level; GenAI service composer to create mega-service
Apache License 2.0
51 stars 110 forks source link

Add vLLM on Ray microservice #245

Closed XinyaoWa closed 2 months ago

XinyaoWa commented 3 months ago

Description

Add vLLM on Ray microservice

Issues

194

Type of change

Dependencies

Tests

chensuyue commented 3 months ago

Please add the e2e test like this one, https://github.com/opea-project/GenAIComps/blob/main/tests/test_llm_summarization_tgi.sh