Add support for generating more than `n_nodes` samples

davmacario / MDI-LLM

Implementation of Model-Distributed Inference for Large Language Models, built on top of LitGPT

MIT License

3 stars 2 forks source link

Add support for generating more than `n_nodes` samples #24

Closed davmacario closed 6 months ago

davmacario commented 6 months ago

This does not only allow to produce more samples and compare evenly different scenarios, but it also improves the time performance, as it reduces the idle/waiting times of the nodes. Closes #10

Also added more verb prints.