Pretraining problem for T5

RedSmallPanda / SSQR

Resources for "Self-Supervised Query Reformulation for Code Search"

2 stars 1 forks source link

Pretraining problem for T5 #2

Open Zxb-QHU opened 6 months ago

Zxb-QHU commented 6 months ago

Follow your steps to pre-train T5, the code won't run. Is there any error in the code in the repository.

Thank you for your answer.

RedSmallPanda commented 6 months ago

Hi, you can first use the pre-train script to train the checkpoint, then put the checkpoint following the guide in readme.

Zxb-QHU commented 6 months ago

Thank you for your response. I have another issue: during training, the GPU is not heavily utilized, but the CPU is maxed out, preventing the program from proceeding. Only when reading CSV files, reading 10 samples allows the training to occur.