search

AFeng-x / SMT

This is an official implementation for "Scale-Aware Modulation Meet Transformer".
https://arxiv.org/abs/2307.08579
MIT License
188 stars 16 forks source link

Can you design a bigger model like smt_huge? #17

Open Geek-lixiang opened 1 year ago

Geek-lixiang commented 1 year ago

Now the biggest model is smt_large, Can you design a bigger model like smt_huge? Thank you very much!

AFeng-x commented 1 year ago

Yes, it's feasible from a design perspective. However, due to the limited availability of GPU machines at the moment, we may temporarily be unable to complete the training of larger models. Nevertheless, this is an experiment we plan to continue in the future. Thank you.