search

IDEA-FinAI / ToG

This is the official github repo of Think-on-Graph. If you are interested in our work or willing to join our research team in Shenzhen, please feel free to contact us by email (xuchengjin@idea.edu.cn)
238 stars 26 forks source link

Using LLama #9

Open alomrani opened 6 months ago

alomrani commented 6 months ago

Hi, I see that you use a local API sever for llama. How do you initialize such a server? Thanks

GasolSun36 commented 6 months ago

We use VLLM: https://github.com/vllm-project/vllm. You can refer to its documentation about server startup.