70x slower generation time with codet5p_2b compared to codet5p_770m

salesforce / CodeT5

Home of CodeT5: Open Code LLMs for Code Understanding and Generation

https://arxiv.org/abs/2305.07922

BSD 3-Clause "New" or "Revised" License

2.65k stars 391 forks source link

Open natashabutt opened 8 months ago

natashabutt commented 8 months ago

My generation time is 70x higher with codet5p_2b compared to codet5p_770m for the same input text. Why is this?