search

salesforce / CodeT5

Home of CodeT5: Open Code LLMs for Code Understanding and Generation
https://arxiv.org/abs/2305.07922
BSD 3-Clause "New" or "Revised" License
2.71k stars 396 forks source link

Is the released pre-trained model including the dual generation pre-training #23

Closed Robin-Y-Ding closed 2 years ago

Robin-Y-Ding commented 2 years ago

Dear authors,

I noticed in the paper you mentioned that you pre-train the T5 model with identifier-aware denoising for 100 epochs and further pre-train with bimodal generation for 50 epochs. I was wondering the released model only includes the first 100 epochs or the whole 150 epochs?

Thanks in advance for your clarification

yuewang-cuhk commented 2 years ago

Hi, the released model is pre-trained with the only identifier-aware denoising for 100 epochs.

Robin-Y-Ding commented 2 years ago

I see, thank you for your response. May I ask whether there is any plan to release the 150-epoch pertaining model? I realized there is the latest checkpoint for multi-lingual code summarization, so I wondered whether there will be an incoming model for NL2code generation?

yuewang-cuhk commented 2 years ago

Yes, we are planning to release another checkpoint for NL2code generation using CodeSearchNet. Pls stay tuned:)

Robin-Y-Ding commented 2 years ago

Great to know! Thanks! Closing the issue.