search

bdqnghi / infercode

[ICSE 2021] - InferCode: Self-Supervised Learning of Code Representations by Predicting Subtrees
89 stars 24 forks source link

Wordering how to use InferCode to predict method names #10

Open ZhangZhuoSJTU opened 3 years ago

ZhangZhuoSJTU commented 3 years ago

Hi all,

Thanks for bringing this excellent work up.

I have played with InferCode for a while and am thinking about how to properly predict method names.

I figured out that InferCode can generate the encoded vectors. Hence, I am thinking maybe we need additionally train a model to predict names using the vectors. May I know whether it is the right way to go? If so, would you mind sharing some ideas/insights about how the architecture of the new model looks like?

Any suggestion would be very much appreciated. Thanks!

bdqnghi commented 3 years ago

hi, thanks for the question.

InferCode is similar to existing pretrained language models, such as Bert, Transformer, etc, which means that once you have a pretrained model on a pretext task, you can fine-tune it, you can read our paper for more details. But if you expect to predict the method name from the vectors, I'm afraid that the reason might not be good.

Avv22 commented 2 years ago

@bdqnghi .

Thank you. Your pretrained model is only for Java please? If we want to use it for multiple programming languages, should we train model from scratch please? If yes, can you refer us back to details how to train model from scratch on a new dataset?

ElchaabiMohamed commented 2 years ago

Hi all,

Thanks for bringing this excellent work up.

I have played with InferCode for a while and am thinking about how to properly predict method names.

I figured out that InferCode can generate the encoded vectors. Hence, I am thinking maybe we need additionally train a model to predict names using the vectors. May I know whether it is the right way to go? If so, would you mind sharing some ideas/insights about how the architecture of the new model looks like?

Any suggestion would be very much appreciated. Thanks!

Hi, thanks for your comment,

can you tell me if you fine-tune infercode successfully ?