Open momo1986 opened 1 year ago
Thanks for your kind comment.
One GPU per machine? Can you explain how many machines you'd like to run it? Or you just want to run it using 1 GPU on 1 machine?
Hi, dear guys, @ghostplant.
I have several different one-gpu machines. To save the computation resource, running the program in a one-gpu machine would be economical for me. Actually, I mainly study some specific properties of MOE. Therefor, if it is OK, as you mentioned, just want to run it using 1 GPU on 1 machine.
If you run it with a one-gpu machine, seems like you need to ensure this GPU memory size is enough to store all 32-expert parameters. The way to convert swin_moe_small_patch4_window12_192_32expert_32gpu_22k.pth
to single-gpu can follow the utility here, where the second example is to merge 32 different checkpoint files into a single checkpoint file and it will be compatible for single gpu to load.
Hi, @ghostplant. Thanks for your guidance. Can this package support run a single-gpu machine to test ImageNet? The user should implement this program manually, or is there a relevant demo? Thanks & Regards! Momo
Hi, dear guys of tutelage team.
I have run the script and do some small modifications.
python -u main_moe.py --cfg configs/swinmoe/swin_moe_small_patch4_window12_192_32expert_32gpu_22k.yaml --data-path /data/user1/junyan/datasets/ImageNet/ImageNet_Val --batch-size 128 --resume checkpoints/swin_moe_small_patch4_window12_192_32expert_32gpu_22k/swin_moe_small_patch4_window12_192_32expert_32gpu_22k.pth
However, I have received the error message:
I have only one gpu. I am not sure whether multiple gpus are essential for this task. Is there a possibility to run it on one gpu? Furthermore, how can I resolve this problem of error?
I am looking forward to your response.
Thanks a lot.
Best Regards!