Regarding the issues that occurred during training

[tobottyx]

Hello, I followed the instructions in the "readme.md" to train the model and I would like to ask about the hardware environment you used during training, including memory and graphics card specifications. I suspect there might be a memory leak issue during my training process, as the memory usage keeps increasing, preventing the training from completing. Thank you.

[aviralchharia]

Hi, how did you fix the memory leak problem?

[tinatiansjz]

Same question. I am also facing the memory leak problem right now, and hope to know the solution.

[tobottyx]

I changed the number of processes read from the data (Num_workers) to match your gpu training speed