Closed Runingtime closed 1 year ago
I noticed that the model architecture from HuggingFace has a shared embedding layer (shared): Embedding(50005, 768, padding_idx=1) whereas Fairseq (used in this repo) does not. Will the shared embedding affect the performance for code-to-text tasks?
(shared): Embedding(50005, 768, padding_idx=1)
Thanks!
I noticed that the model architecture from HuggingFace has a shared embedding layer
(shared): Embedding(50005, 768, padding_idx=1)
whereas Fairseq (used in this repo) does not. Will the shared embedding affect the performance for code-to-text tasks?Thanks!