Experiment with Lightning fabric, reproduce speed improvement from: https://lightning.ai/pages/community/finetuning-falcon-efficiently/

jina-ai / jerboa

LLM finetuning

Apache License 2.0

41 stars 4 forks source link

Experiment with Lightning fabric, reproduce speed improvement from: https://lightning.ai/pages/community/finetuning-falcon-efficiently/ #86

Closed sebastian-weisshaar closed 1 year ago

sebastian-weisshaar commented 1 year ago

The experiments in the article have a 8x speedup over our current setup. We want to reproduce these results and understand what elements of their code we can use.