[feat]: swap client/server nodes

davmacario / MDI-LLM

Implementation of Model-Distributed Inference for Large Language Models, built on top of LitGPT

MIT License

3 stars 2 forks source link

Closed davmacario closed 8 months ago

davmacario commented 8 months ago

Now the nodes act as clients for the following ones, and as servers for the previous ones.
Improved stability and efficiency by removing the waiting delay when message queue is empty