-
Hi,
Thanks for your great work.
I would like to train the model using multiple GPUs but I receive this error:
" _RuntimeError: CUDA error: invalid device ordinal CUDA kernel errors might be a…
-
Hi, Liu. Thanks for sharing your work. Now, I meet a problem when training the simple-IAM with multi-gpus. nn.DataParallel works when training the prm classification networks. However, the training p…
-
### 🚀 The feature, motivation and pitch
As we can see, Google Gemini can support up to million tokens and to serve longer context length, we have to do context parallelism, which means, split the i…
-
hi @mjq11302010044,
thanks for your great work.
i have a question, does the code support multi-gpus training?
thanks.
-
Hi, team, I would like to know how to load and dump a sharded embedding collection via `state_dict`. Basically
1. How many files should I save? Should each rank have an exclusive sharding file or o…
-
I have implmented a multi-gpus version.
1>But I am not sure how to evaluate reproduce to be successful for the search process.What's the correct evaluation accuracy in the training process?
2>And I…
-
I have 2x 4090, is it possible to have this model run across multiple gpus, or offload the kv cache to another cuda device? I'm fairly certain its the kv_cache being offloaded to cpu thats causing it …
-
HI @dstamoulis Thanks for your code!
I have modified TPU setting into GPU, like tf.estimator.Estimator, tf.estimator.RunConfig, and single GPU works.
However, when I apply "MirroredStrategy" into t…
-
How to train easyanimate with multi machines multi gpus?
-
Hi, can you tell me how to use multi gpus to train the model? When I runing "python main_pretrain_cpc.py", it only uses the first gpu.
Thanks a lot.