search

salesforce / CodeT5

Home of CodeT5: Open Code LLMs for Code Understanding and Generation
https://arxiv.org/abs/2305.07922
BSD 3-Clause "New" or "Revised" License
2.68k stars 394 forks source link

Fined-tuned checkpoints -> Code clone detection #55

Closed Chinmayrane16 closed 2 years ago

Chinmayrane16 commented 2 years ago

Hi,

I am hoping to reproduce the results on code clone detection task. This might seem like a silly question but the fined-tuned checkpoints released doesn't include the RobertaClassificationHead parameters, right? I am able to load only the T5ForConditionalGeneration model using the provided checkpoints for the task.

So, how do I go about loading the entire CloneModel?

yuewang-cuhk commented 2 years ago

Hi, thanks for pointing out this. We will update this finetuned checkpoint with the RobertaClassificationHead parameters.

sethun commented 2 years ago

Hi!

Not sure if it is too-early, I downloaded the latest checkpoint (Uploaded an hour ago). Seems there is some-mismatch between ClonedModel's shape and checkpoint's shape

RuntimeError: Error(s) in loading state_dict for CloneModel: size mismatch for encoder.shared.weight: copying a param with shape torch.Size([32000, 768]) from checkpoint, the shape in current model is torch.Size([32100, 768]). size mismatch for encoder.encoder.embed_tokens.weight: copying a param with shape torch.Size([32000, 768]) from checkpoint, the shape in current model is torch.Size([32100, 768]). size mismatch for encoder.decoder.embed_tokens.weight: copying a param with shape torch.Size([32000, 768]) from checkpoint, the shape in current model is torch.Size([32100, 768]). size mismatch for encoder.lm_head.weight: copying a param with shape torch.Size([32000, 768]) from checkpoint, the shape in current model is torch.Size([32100, 768]).

yuewang-cuhk commented 2 years ago

Hi, we just updated this finetuned checkpoint yesterday. You can resolve this via adding this line. Following the instructions here, you will be able to reproduce the results of: [best-f1] test-f1: 0.9500, precision: 0.9526, recall: 0.9474. This result is different from the reported one in the paper due to that we report micro-f1 instead of macro-f1 following GraphCodeBERT.