search

salesforce / CodeT5

Home of CodeT5: Open Code LLMs for Code Understanding and Generation
https://arxiv.org/abs/2305.07922
BSD 3-Clause "New" or "Revised" License
2.75k stars 403 forks source link

How to get embedding for javascript and python code snippet? #62

Closed smith-co closed 1 year ago

smith-co commented 2 years ago

I have a couple of questions:

a) How can I use CodeT5 to extract embedding for JavaScript and Python code? b) Can I feed incomplete code JavaScript and Python snippet to extract embedding? Or the code snippet needs to be complete? c) Have anyone used CodeT5 to perform code to code search?

yuewang-cuhk commented 2 years ago

Hi, for embedding exaction with CodeT5, we suggest to follow the BART manner by feeding the sequences to both encoder and decoder. Then you can employ either the last decoder state or the average pooling of all decoder states as the sequence embedding.

For code search, we did not try it yet. You can refer to this UniXcoder paper which includes such evaluation for the moment.

TejaswiniiB commented 2 years ago

Hi @yuewang-cuhk Does code-t5 give proper embeddings for python code with multiple methods as well? Or only for the code with single method?

yuewang-cuhk commented 1 year ago

HI TejaswiniiB, we did not test it but believe it should give proper embeddings.