1.Modularize embedding component.
2.Change the model config file in the models folder and rearrange them.
3.Change the convert function in the scripts
4.Eliminate deepspeed support.
5.Change two things in .github/workflows/github-actions.yml:
a.python version from 3.6.13 -> 3.7.13 since the latest ubuntu doesn't support python 3.6.
b. change the default model config for the instruction of pretraining encoder_decoder model from models/encoder_decoder_config.json -> models/transformer/base_config.json, since models/encoder_decoder_config.json is removed.
1.Modularize embedding component. 2.Change the model config file in the models folder and rearrange them. 3.Change the convert function in the scripts 4.Eliminate deepspeed support. 5.Change two things in .github/workflows/github-actions.yml: a.python version from 3.6.13 -> 3.7.13 since the latest ubuntu doesn't support python 3.6. b. change the default model config for the instruction of pretraining encoder_decoder model from models/encoder_decoder_config.json -> models/transformer/base_config.json, since models/encoder_decoder_config.json is removed.