Develop - Githubissues

davmacario / MDI-LLM

Implementation of Model-Distributed Inference for Large Language Models, built on top of LitGPT

MIT License

3 stars 2 forks source link

Closed davmacario closed 6 months ago

davmacario commented 6 months ago

Changes:

Use threading.Event to wait for messages in the input queue at each node (no busy waiting anymore)
Reduce memory usage by deleting unused variables and possible copies
(Attempt to) improve program to monitor memory usage - Not working
Add programs to produce plots for result comparison
Update readme and add images
Fix minor bugs/issues