How much memory is required ?

[breakenknife]

How much memory is required to run convert-hf-to-ggml.py and run inference locally?

[NouamaneTazi]

• convert-hf-to-ggml.py : By offloading the model's weights to disk, you only need to be able to fit a single weight in your ram which shouldn't be an issue
• inference: You can check https://github.com/bigcode-project/starcoder.cpp#quantizing-the-models for the memory required per model