search

davmacario / MDI-LLM

Implementation of Model-Distributed Inference for Large Language Models, built on top of LitGPT
MIT License
3 stars 2 forks source link

[bug]: reduce memory usage, the correct way #5

Open davmacario opened 7 months ago

davmacario commented 7 months ago

To be corrected in the code: instead of using var = None to clear it, invoke the garbage collector and delete the variable data.

import gc
del var
gc.collect()
davmacario commented 7 months ago

Done, but not providing the expected results; will keep open - see: karpathy/nanoGPT#436.