search

lyogavin / airllm

AirLLM 70B inference with single 4GB GPU
Apache License 2.0
5.28k stars 423 forks source link

Ramdisk #155

Open HennethAnnun opened 4 months ago

HennethAnnun commented 4 months ago

Hi, did anyone try running AirLLM with any model directly from RAM with a ramdisk? Seems like an obvious choice. I am curious how would llama3 70B perform.