search

hpcaitech / EnergonAI

Large-scale model inference.
Apache License 2.0
630 stars 90 forks source link

OPT inference generate example #102

Open virgulvirgul opened 2 years ago

virgulvirgul commented 2 years ago

Hi, is there any generate example for OTP models?

dujiangsu commented 2 years ago

Hi， you can follow https://github.com/dujiangsu/EnergonAI/blob/feature/opt-175b/energonai/utils/checkpointing_opt.py to load parameters of opt model. As for the generate example, you can refer to the hf_gpt2 example.