Does this framework support SFT?

databricks / megablocks

Apache License 2.0

1.11k stars 154 forks source link

Does this framework support SFT? #90

Open banksy23 opened 5 months ago

tgale96 commented 5 months ago

Hi! MegaBlocks isn't a standalone training framework, but it's relatively easy to use from any framework. We use Megatron-LM and have a fork of it with MegaBlocks support. You could also use MegaBlocks from another framework like HuggingFace, for example.

alpayariyak commented 3 months ago

Do you have SFT scripts? And Hyperparameters that you used to fine-tune the instruct version of your model? It would mean the world for the OS community!

Edit: created an issue for this #99