search

salesforce / CodeT5

Home of CodeT5: Open Code LLMs for Code Understanding and Generation
https://arxiv.org/abs/2305.07922
BSD 3-Clause "New" or "Revised" License
2.68k stars 394 forks source link

Minimal snips to run CodeT5 on each task #73

Closed rocco-fortuna closed 1 year ago

rocco-fortuna commented 1 year ago

Hello, I have spent quite some time trying to run the fine-tuned model you kindly provided, on the clone detection task, with no luck. Could you provide minimal python scripts to load and run the model on each of the downstream tasks, as was done for ntp and summarization on 🤗?

Thanks a lot!

rocco-fortuna commented 1 year ago

I am happy to contribute by providing such scripts, but need guidance to make your model work. Your finetuned_models_clone_codet5_base.bin model only ever predicts code pairs in the test set are not clones for me, which yields 0 F1 score, not quite the 97% it should reach according to the paper.

yuewang-cuhk commented 1 year ago

Hi, we've shared the instructions to reproduce the results using our released checkpoints here. Please try it or you can rerun the finetuning experiments. If both options still do not work, please share more detailed logs for us to understand the issue.