Right now, GPT4All only utilizes 1 GPU so for machines with multiple GPU's, it blocks them from having access to higher parameter count models to use.
Steps to Reproduce
2x RTX 3090 installed
Download llama-3-70b
Try and load the model and watch it load one GPU with 24gb and then crash, 2nd GPU is not utilized
Expected Behavior
In rigs where there are multiple GPU's the app should be able to split the models across them - enabling users to have access of higher parameter count models.
Bug Report
Right now, GPT4All only utilizes 1 GPU so for machines with multiple GPU's, it blocks them from having access to higher parameter count models to use.
Steps to Reproduce
Expected Behavior
In rigs where there are multiple GPU's the app should be able to split the models across them - enabling users to have access of higher parameter count models.
Your Environment