Bug when finetuning Mistral 7B

brevdev / notebooks

Collection of notebook guides created by the Brev.dev team!

MIT License

1.62k stars 275 forks source link

Bug when finetuning Mistral 7B #6

Closed MLlove0402 closed 3 months ago

MLlove0402 commented 9 months ago

Hello, Thanks you for great work I'm follow your code mistral-finetune-own-data.ipynb, but when the code run at: accelerator = Accelerator(fsdp_plugin=fsdp_plugin) model = accelerator.prepare_model(model)

It throw an error: ValueError: Must flatten tensors with uniform requires_grad when use_orig_params=False

How can i fix it. Thanks you

linpan commented 8 months ago

ValueError: Can't find 'adapter_config.json' when gguf-export.ipynb