Hi I was wondering if anyone has been able to get LightSeq working on GCP.
I have been trying for the last few weeks and still have not had any luck.
I have been able to get training of small models working but not consistently, eventually there is a CUDA out of memory error or an infinite timeout loop that I run into.
I am thinking these issues arise from the fact that this was written in CUDA <11, and all preexisting images for GCP are CUDA >11.
Please let me know if anyone has a machine configuration and startup script that can get the simple translation task transformer example to work.
Anychnn
commented
2 days ago
Hi I was wondering if anyone has been able to get LightSeq working on GCP.
I have been trying for the last few weeks and still have not had any luck. I have been able to get training of small models working but not consistently, eventually there is a CUDA out of memory error or an infinite timeout loop that I run into. I am thinking these issues arise from the fact that this was written in CUDA <11, and all preexisting images for GCP are CUDA >11. Please let me know if anyone has a machine configuration and startup script that can get the simple translation task transformer example to work.
Thanks in advance.