Can't Generate using Finetuned checkpoint

[danzeeeman]

I pass in the correct base model and the folder for the checkpoint for the lora but it says

ValueError: Can't find 'adapter_config.json'

[jb-01]

generate.py should not be given the folder to a checkpoint. Your finetuned model folder should contain an adapter_config.json file.

For example:

/path/to/finetuned_model/ ├─ adapter_config.json ├─ checkpoint-200/ │ ├─ optimizer.pt │ ├─ ...

You need to pass this entire directory to generate.py: --lora_weights 'path/to/finetuned_model'

[danzeeeman]

'adapter_config.json' doesn't get generated?

[jb-01]

Are you using a custom finetuned model or a HF checkpoint?

[danzeeeman]

yahma/llama-7b-hf is the base model

[jb-01]

What are you using for --lora_weights ?

python generate.py \
    --load_8bit \
    --base_model 'decapoda-research/llama-7b-hf' \
    --lora_weights 'tloen/alpaca-lora-7b'

[danzeeeman]

the path to my custom lora I'm training that will take 250hr to finish and want to check the 200/400/600/800 check points

[jb-01]

Have you tried temporarily moving the other checkpoints out of your model's folder to test each one individually?

[danzeeeman]

I'm saying adapter_config.json doesn't get generated at the start of finetuning as expected.

[jb-01]

adapter_config.json doesn't get generated until after training is complete.

Here is a template for an adapter_config.json file in case you wanted to manually test earlier checkpoints. Replace with your personal parameters.

{
  "base_model_name_or_path": "yahma/llama-7b-hf",
  "bias": "none",
  "enable_lora": null,
  "fan_in_fan_out": false,
  "inference_mode": true,
  "init_lora_weights": true,
  "lora_alpha": 16,
  "lora_dropout": 0.05,
  "merge_weights": false,
  "modules_to_save": null,
  "peft_type": "LORA",
  "r": 8,
  "target_modules": [
    "q_proj",
    "v_proj"
  ],
  "task_type": "CAUSAL_LM"
}

[danzeeeman]

Shouldn’t that get created at the start of training?

On Mon, May 22, 2023 at 4:00 PM Joshua Belofsky @.***> wrote:

adapter_config.json doesn't get generated until after training is complete.

Here is a template for an adapter_config.json file in case you wanted to manually test earlier checkpoints. Replace with your personal parameters.

{ "base_model_name_or_path": "yahma/llama-7b-hf", "bias": "none", "enable_lora": null, "fan_in_fan_out": false, "inference_mode": true, "init_lora_weights": true, "lora_alpha": 16, "lora_dropout": 0.05, "merge_weights": false, "modules_to_save": null, "peft_type": "LORA", "r": 8, "target_modules": [ "q_proj", "v_proj" ], "task_type": "CAUSAL_LM" }

— Reply to this email directly, view it on GitHub https://github.com/tloen/alpaca-lora/issues/469#issuecomment-1557880405, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFPVTD2SQ7Z2G4VXV3IYB3XHPAWDANCNFSM6AAAAAAYJRC4NI . You are receiving this because you authored the thread.Message ID: @.***>

-- Sent from Gmail Mobile #autocorrect