Support for memmap'd gguf files?

nonnonstop / comfyui-faster-loading

Speed up the loading of checkpoints with ComfyUI.

Creative Commons Zero v1.0 Universal

4 stars 2 forks source link

Support for memmap'd gguf files? #1

Open brendanhoar opened 3 weeks ago

brendanhoar commented 3 weeks ago

Looks like those of us with storage that dislikes many small reads are having the same symptoms for gguf files, since the gguf_reader.py library also uses memmap, similar to how the safetensors library does...

Is this something you'd take a look at?

Amit30swgoh commented 2 weeks ago

now sure how to use it