Open Chting opened 1 year ago
The original Meta's repo works with A100s as well.
The original Meta's repo works with A100s as well.
It stipulates that 65B must use 8GPU
I'm also trying to figure out how to run with 2 gpus
The original Meta's repo works with A100s as well.
how to run example-chat.py with 8A100?
I'm also trying to figure out how to run with 2 gpus
If you succeed, please tell me
Reshard the model for the number of GPUs you have: https://gist.github.com/benob/4850a0210b01672175942203aa36d300
Resharding the larger models to one file can improve load times in general.
@Chting @wgimperial @fmeres now you may try to run HF version on a more than one GPU's.
@Chting @wgimperial @fmeres now you may try to run HF version on a more than one GPU's.
Thank you very much
I'm testing 65B. One A100 is too slow. I want to use two or four