search

alpa-projects / alpa

Training and serving large-scale neural networks with auto parallelization.
https://alpa.ai
Apache License 2.0
3.05k stars 353 forks source link

Will OPT-IML-175B be supported? #868

Open JingfengYang opened 1 year ago

JingfengYang commented 1 year ago

System information

Describe the new feature and the current behavior/state Alpa supports OPT-175B currently. But there is a recent model of OPT-IML-175B, which is better than OPT released by Meta. Gthub link: https://github.com/facebookresearch/metaseq/tree/main/projects/OPT-IML .

Will this change the current API? How? No

Describe alternatives you've considered I have tried OPT-175B

Additional context

jellyheadandrew commented 1 year ago

+1

zhisbug commented 1 year ago

Any forms of contributions about OPT-IML are welcome

GanjinZero commented 1 year ago

+1

skyz8421 commented 1 year ago

+1

KazutoshiShinoda commented 1 year ago

+1

larekrow commented 1 year ago

+1

dlzou commented 1 year ago

I investigated this and found that only minimal config changes are needed to support OPT-IML in the llm_serving example, since it has the same architecture as OPT. I added those changes in #894 and confirmed that textgen.py runs for up to OPT-IML-Max-30B on 8x Tesla P100s.

larekrow commented 1 year ago

I am looking for the conversion scripts/steps for the OPT-IML-175B weights, similar to the conversion for the OPT-175B weights. The metaseq OPT-175B weights are given as 992 FSDP shards while the metaseq OPT-IML-175B weights are given as 16 TPs, so I am not quite sure how I could run step 2.