search

salesforce / CodeT5

Home of CodeT5: Open Code LLMs for Code Understanding and Generation
https://arxiv.org/abs/2305.07922
BSD 3-Clause "New" or "Revised" License
2.68k stars 394 forks source link

Fined-tuned checkpoints -> defect detection #60

Closed Kamel773 closed 1 year ago

Kamel773 commented 2 years ago

Hi CodeT5 team,

I want to load the checkpoints " defect_codet5_base.bin", but I am getting the error below:

Traceback (most recent call last): File "/notebooks/CodeT5//run_defect.py", line 377, in <module> main() File "/notebooks/CodeT5//run_defect.py", line 354, in main model.load_state_dict(torch.load(link)) File "/usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py", line 1604, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for DefectModel: Missing key(s) in state_dict: "encoder.shared.weight", "encoder.encoder.embed_tokens.weight", "encoder.encoder.block.0.layer.0.SelfAttention.q.weight", "encoder.encoder.block.0.layer.0.SelfAttention.k.weight", "encoder.encoder.block.0.layer.0.SelfAttention.v.weight", "encoder.encoder.block.0.layer.0.SelfAttention.o.weight", "encoder.encoder.block.0.layer.0.SelfAttention.relative_attention_bias.weight",

yuewang-cuhk commented 1 year ago

Hi, the released checkpoint of "defect_codet5_base.bin" for defect detection is a generator to directly predict the binary label from the decoder, while "run_defect.py" is an example script to approach this task as a classification task. So you should employ the "run_gen.py" instead of "run_defect.py" to load this checkpoint.