heurist-network / miner-release

Stable Diffusion and LLM miner for Heurist
Other
40 stars 34 forks source link

perf: update llm starter script to allow distributed inference #31

Open EntroSanity opened 4 months ago

EntroSanity commented 4 months ago

This pull request updates validateVram function in llm starter script to check for total available vram when user intends to deploy large models across multiple gpus; it comes with a usage example in README