search

microsoft / CodeBERT

CodeBERT
MIT License
2.15k stars 442 forks source link

type embedding of unixcoder #272

Open xieexiaotuzi opened 1 year ago

xieexiaotuzi commented 1 year ago

Hi, Thanks for your great job.

I would like to ask why the type of embedding of unixcoder is Embedding(10, 768) ?

Best, Lucas.

guoday commented 1 year ago

In UniXcoder, we have created 10 types for future usage. Each type has a dimension of 768, resulting in a type embedding size of (10, 768). However, at present, we are only utilizing the embedding for the first type (type 0) and not utilizing the embeddings for the remaining types.