Closed chuanli11 closed 1 year ago
Fine-tuning Falcon LLMs with data parallelization.
Tested on A100 80G SXM4 and achieves nearly perfect scaling from 1x to 8x GPUs.
Fine-tuning Falcon LLMs with data parallelization.
Tested on A100 80G SXM4 and achieves nearly perfect scaling from 1x to 8x GPUs.