[LM Pipeline] learn to use different LM tools to generate LM finetuning pipeline

[leeeizhang]

Help users to quickly build an efficient LM training & finetuning pipeline, and also give the multi-gpu parallel codes.

MLE-agent could learn to use:

• parallel tools: Deepspeed, Megatron, FSDP
• compile tools: Torch.compile, TensorRT, TVM
• algorithms: lora, kv-cache, falcon, xAttention, and etc.

[HuaizhengZhang]

https://github.com/NVIDIA/NeMo

how about addingnemo also

[HuaizhengZhang]

https://github.com/OpenAccess-AI-Collective/axolotl