search

ml-explore / mlx-examples

Examples in the MLX framework
MIT License
5.79k stars 823 forks source link

Impossible to fuse Mistral-7b-Instruct-v0.2 after finetuning #559

Open antoinelemor opened 5 months ago

antoinelemor commented 5 months ago

Hi everyone,

I'm unable to fuse the model after finetuning, I got this error. Can someone please help? All paths are correct and the adapter works just fine.

antoine@Mac-Studio lora % python fuse.py --model "/Users/antoine/Documents/GitHub/EVD.COVID_ANALYSIS/EVD.COVID_ANALYSIS/Models.nosync/Mistral/Mistral-7B-Instruct-v0.2/" --save-path "/Users/antoine/Documents/GitHub/EVD.COVID_ANALYSIS/EVD.COVID_ANALYSIS/Models.nosync/Mistral/fine-tuned/" --adapter-file "/Users/antoine/Documents/GitHub/EVD.COVID_ANALYSIS/EVD.COVID_ANALYSIS/Models.nosync/Mistral/adapters/adapters.npz" --de-quantize Loading pretrained model Traceback (most recent call last): File "/Users/antoine/Documents/GitHub/EVD.COVID_ANALYSIS/EVD.COVID_ANALYSIS/mlx-examples-main-2/lora/fuse.py", line 55, in model, tokenizer, config = utils.load(args.model) ^^^^^^^^^^^^^^^^^^^^^^ File "/Users/antoine/Documents/GitHub/EVD.COVID_ANALYSIS/EVD.COVID_ANALYSIS/mlx-examples-main-2/lora/utils.py", line 154, in load raise FileNotFoundError("No safetensors found in {}".format(model_path)) FileNotFoundError: No safetensors found in /Users/antoine/Documents/GitHub/EVD.COVID_ANALYSIS/EVD.COVID_ANALYSIS/Models.nosync/Mistral/Mistral-7B-Instruct-v0.2

awni commented 5 months ago

The --model flag should point to the original model (hugging face repo or local path) that you fine tuned with.

  1. Could you share the output of: 
    ls /Users/antoine/Documents/GitHub/EVD.COVID_ANALYSIS/EVD.COVID_ANALYSIS/Models.nosync/Mistral/Mistral-7B-Instruct-v0.2/
  2. Could you share the command you used to fine-tune?