Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any open-source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop.
请问跑minicpm-llama3-v-2_5(int4)支持并发调用接口么?2个及以上并发调用就报错了,单个没有问题。。。。
![微信截图_20240619180359](https://github.com/xorbitsai/inference/assets/167763677/a91e48c4-c31e-48b9-917b-303ceb67cef1)
需要怎么样才能并发呢?目前是一台物理机 24G 显卡,虽然资源不多,但希望能够实现起码2个并发吧~~