Open pjyi2147 opened 4 days ago
I am trying to use the direct output of MLCEngine class but I do not have any clue how to get the output based on the MLC-LLM docs. Is there any similar API to MLCEngine such as the examples in MLC-LLM?
MLCEngine
https://docs.vllm.ai/en/latest/getting_started/quickstart.html https://sgl-project.github.io/backend/offline_engine_api.html
I am trying to use the direct output of
MLCEngine
class but I do not have any clue how to get the output based on the MLC-LLM docs. Is there any similar API toMLCEngine
such as the examples in MLC-LLM?https://docs.vllm.ai/en/latest/getting_started/quickstart.html https://sgl-project.github.io/backend/offline_engine_api.html