[PROPOSAL]: Publish the reproduction results on some mainstream tasks(both image and text).

hpcaitech / ColossalAI

Making large AI models cheaper, faster and more accessible

https://www.colossalai.org

Apache License 2.0

38.8k stars 4.35k forks source link

[PROPOSAL]: Publish the reproduction results on some mainstream tasks(both image and text). #2831

Open flymark2010 opened 1 year ago

flymark2010 commented 1 year ago

Proposal

For now, I have not found any reproduction results using ColossalAI. It's importance to make the community more trust by having some reproduction on some mainstream image and text tasks with both single GPU and multi-node GPUs in difference distributed modes.

Self-service

[ ] I'd be willing to do some initial work on this proposal myself.

kurisusnowdeng commented 1 year ago

It's a great suggestion. We once make few efforts to present SOTA results using ColossalAI, which are in our benchmark repo. Sadly, we do not have enough resources to complete more results other than ViT so far. So, you are very very welcome to help us! BTW, bert and gpt2 on openwebtext are already under evaluation, which can be released as soon as possible. We appreciate it if you could contribute more to this project. To discuss the details, you are welcome to join our wechat group or slack channel.

FrankLeeeee commented 1 year ago

Good suggestions. That is part of our plan. We are releasing some future development plans for this project in the community soon and hopefully the benchmark can be refined with this plan.