search

bofenghuang / vigogne

French instruction-following and chat models
Apache License 2.0
497 stars 47 forks source link

ERROR: The function received no value for the required argument: lora_model_name_or_path #2

Open pereconteur opened 1 year ago

pereconteur commented 1 year ago

Hi,

I would like to combine your model with Alpaca's. I have installed everything, and made a git clone of the lora model repository at 7B.

Only, despite the path to it no way to continue, am I missing something ?

python vigogne/scripts/export_state_dict_checkpoint.py \
    --base_model_name_or_path <../alpaca.cpp/ggml-alpaca-7b-q4.bin> \
    --lora_model_name_or_path "vigogne-lora-7b" \
    --output_dir ./models/7B
ERROR: The function received no value for the required argument: lora_model_name_or_path
Usage: export_state_dict_checkpoint.py BASE_MODEL_NAME_OR_PATH LORA_MODEL_NAME_OR_PATH OUTPUT_DIR

For detailed information on this command, run:
  export_state_dict_checkpoint.py --help

Tree of Vigogne-Lora : 

vigogne-lora-7b
├── README.md
├── adapter_config.json
├── adapter_model.bin
├── runs
│   └── Mar26_00-12-50_koios.zaion.ai
│       ├── 1679785970.575404
│       │   └── events.out.tfevents.1679785970.koios.zaion.ai
│       └── events.out.tfevents.1679785970.koios.zaion.ai
├── tokenizer.model
└── vigogne_logo.png
bofenghuang commented 1 year ago

Hi @pereconteur!

Unfortunately, you cannot merge the Vigogne-LoRA weights with the Alpaca model.

The base_model_name_or_path parameter should include model files for the LLaMA model in Hugging Face's format.

Thank you for asking. I will update the README to clarify this!

pereconteur commented 1 year ago

Salut je suppose que tu comprend le francais.

J'ai téléchargé le fichier : llama-7 B-hf, qui à mon avis est le modèle de Llama au format Hugging Face's : 

llama-7b-hf
├── LICENSE
├── README.md
├── config.json
├── generation_config.json
├── gitattributes
├── pytorch_model-00001-of-00033.bin
├── pytorch_model-00002-of-00033.bin
├── pytorch_model-00003-of-00033.bin
├── pytorch_model-00004-of-00033.bin
├── pytorch_model-00005-of-00033.bin
├── pytorch_model-00006-of-00033.bin
├── pytorch_model-00007-of-00033.bin
├── pytorch_model-00008-of-00033.bin
├── pytorch_model-00009-of-00033.bin
├── pytorch_model-00010-of-00033.bin
├── pytorch_model-00011-of-00033.bin
├── pytorch_model-00012-of-00033.bin
├── pytorch_model-00013-of-00033.bin
├── pytorch_model-00014-of-00033.bin
├── pytorch_model-00015-of-00033.bin
├── pytorch_model-00016-of-00033.bin
├── pytorch_model-00017-of-00033.bin
├── pytorch_model-00018-of-00033.bin
├── pytorch_model-00019-of-00033.bin
├── pytorch_model-00020-of-00033.bin
├── pytorch_model-00021-of-00033.bin
├── pytorch_model-00022-of-00033.bin
├── pytorch_model-00023-of-00033.bin
├── pytorch_model-00024-of-00033.bin
├── pytorch_model-00025-of-00033.bin
├── pytorch_model-00026-of-00033.bin
├── pytorch_model-00027-of-00033.bin
├── pytorch_model-00028-of-00033.bin
├── pytorch_model-00029-of-00033.bin
├── pytorch_model-00030-of-00033.bin
├── pytorch_model-00031-of-00033.bin
├── pytorch_model-00032-of-00033.bin
├── pytorch_model-00033-of-00033.bin
├── pytorch_model.bin.index.json
├── special_tokens_map.json
├── tokenizer(1).model
└── tokenizer_config.json

Dit moi si c'est bien ca.

Si c'est ca, j'ai effectué la commande suivante : 

python export_state_dict_checkpoint.py \
    --base_model_name_or_path <../../llama-7b-hf/> \
    --lora_model_name_or_path "bofenghuang/vigogne-lora-7b" \
    --output_dir vigogne/scripts/./models/7B \
    --checkpoint_size "7b"

et j'ai cette erreur qui survient : 

Fatal Python error: init_sys_streams: <stdin> is a directory, cannot continue
Python runtime state: core initialized

Current thread 0x00000001fdebdb40 (most recent call first):
  <no Python frame>

Une idée pour m'aider stp, j'adorerais utilise une version francaise.

Merci ❤️

bofenghuang commented 1 year ago

HI @pereconteur ,

Désolé pour la réponse tardive.

Pourrais-tu réssayer avec la commande suivante ?

python export_state_dict_checkpoint.py \
    --base_model_name_or_path ../../llama-7b-hf \
    --lora_model_name_or_path "bofenghuang/vigogne-lora-7b" \
    --output_dir vigogne/scripts/./models/7B \
    --checkpoint_size "7b"
pereconteur commented 1 year ago

Salut,

C'est bon j'ai réussi à le faire fonctionner, enfaite je n'avais pas les bons poids pour Llama et pour ceux que ça intéresse j'ai trouvé les poids ici :

Et il faut que le dossier à 7B correspond à ces hashes :

7935c843a25ae265d60bf4543b90bfd91c4911b728412b5c1d5cff42a3cd5645  ./models/7B/checklist.chk
700df0d3013b703a806d2ae7f1bfb8e59814e3d06ae78be0c66368a50059f33d  ./models/7B/consolidated.00.pth
7e89e242ddc0dd6f060b43ca219ce8b3e8f08959a72cb3c0855df8bb04d46265  ./models/7B/params.json

Après ça tout roule !

Merci pour ton repository !

pereconteur commented 1 year ago

Tu pourrais m'expliquer comment t'a fait pour la fine tuner ? Et peut-on fine tuner un modèle avec un fichier .json avec des instructions, input et output ?

esokullu commented 1 year ago

Hello! Here's the error I get with export_state_dict_checkpoint.py

on CPU; Loading checkpoint shards: 100% ... then: RuntimeError: shape '[32, 2, 64, 4096]' is invalid for input of size 26214400

on GPU; Loading checkpoint shards: 100% File "scripts/export_state_dict_checkpoint.py", line 128, in fire.Fire(main) File "/opt/conda/lib/python3.8/site-packages/fire/core.py", line 141, in Fire component_trace = _Fire(component, args, parsed_flag_args, context, name) File "/opt/conda/lib/python3.8/site-packages/fire/core.py", line 475, in _Fire component, remaining_args = _CallAndUpdateTrace( File "/opt/conda/lib/python3.8/site-packages/fire/core.py", line 691, in _CallAndUpdateTrace component = fn(*varargs, **kwargs) File "scripts/export_state_dict_checkpoint.py", line 115, in main new_state_dict[new_k] = unpermute(v) File "scripts/export_state_dict_checkpoint.py", line 71, in unpermute return w.view(n_heads, 2, dim // n_heads // 2, dim).transpose(1, 2).reshape(dim, dim) RuntimeError: shape '[32, 2, 64, 4096]' is invalid for input of size 26214400

this is on 13B.

I get the same error on both. No idea what's going on!?

esokullu commented 1 year ago

here's what chatgpt says :)

The error you're encountering is a shape mismatch error when reshaping the tensor in both CPU and GPU environments. The script is trying to reshape the tensor to [32, 2, 64, 4096], which would result in a tensor of size 16,777,216 elements. However, the input tensor has a size of 26,214,400 elements, which is not compatible with the target shape.

Here are a few suggestions to help debug the issue:

Verify the dimensions: Double-check the dimensions and shapes of the tensors you are working with. Make sure that the dimensions of the input tensor are correct and correspond to the expected dimensions for the model you are using.

Inspect the script: Look closely at the export_state_dict_checkpoint.py script, specifically at the unpermute function. Check if there are any hard-coded values that may be causing the shape mismatch. If you find any, update them to the appropriate values based on the model you are using.

Check the model architecture: Make sure you are using the correct model architecture (13B) and that the script is compatible with this specific architecture.

Update the script (if necessary): If the script you are using is outdated or not compatible with the 13B model, try to find an updated version of the script or modify it to handle the model correctly.

Verify the checkpoint: Make sure that the checkpoint file you are loading is not corrupted or mismatched. If you are unsure, try downloading the checkpoint file again.

So I guess it's an issue with 13B? I've completed all steps prior successfully. Will try again with a smaller model, if that is not an error with the script.

bofenghuang commented 1 year ago

Hi @esokullu,

It seems like it used the parameters of 7B model to export a 13B. Did you specify the --base_model_size 13B when running export_state_dict_checkpoint.py? For more information please refer to this section.

bofenghuang commented 1 year ago

Hi @pereconteur,

Yes, you just need to format your data to a JSON file with instruction, input, and output, then launch the fine-tuning using this command

pereconteur commented 1 year ago

Ok et c'est possible de fine-tunner avec un Macbook Pro M1 ? Car j'ai cette erreur lorsque j'essaie ta commande : 

ValueError: FP16 Mixed precision training with AMP or APEX (`--fp16`) and FP16 half precision evaluation (`--fp16_full_eval`) can only be used on CUDA devices.
bofenghuang commented 1 year ago

I've never tried with M1. But maybe you can take a look at here

pereconteur commented 1 year ago

Ok merci, je vais tester ça et je te tiens au courant !

J'ai fait une vidéo Youtube sur ta solution je te mets le lien ci-dessous si tu veux la visionner, en espérant t'avoir fait une belle pub sur tes travaux.

https://www.youtube.com/watch?v=BBf5h0HCFMY&t=292s

bofenghuang commented 1 year ago

Merci beaucoup pour cette vidéo ! C'est super clair et précis !

Do you mind if I put the link in the README?

pereconteur commented 1 year ago

Non au contraire ! Ça me fait super plaisir de pouvoir contribuer à ton projet !

Je te pose toute ces questions parce que ma prochaine vidéo portera sur la fine-tuning, et j'aimerais en montrer l'exemple avec ton repository. Mais vu que j'ai un mac, et donc pas de carte Nvidia, je ne peux pas utiliser CUDA pour la fine-tuning et donc je cherche une solution pour le faire sur mon mac tu comprend ?

bofenghuang commented 1 year ago

Unfortunately, running PyTorch on M1 may be unfeasible because it is primarily optimized for CUDA. Have you considered using a GPU cloud instead?

pereconteur commented 1 year ago

Ok,

I have a request. To make my video tutorial on fine-tuning your repo, I need to show how to do it with your finetune.py file, would it be possible for you to film your screen doing the steps, and send them to me, for the purpose of the video? I would like to show how to modify the json file and then teach it what to put in the json file and then show the result.

For example, I wanted to put in the Json file the instruction: "Donne moi le top 10 du ballon d'or 2016 (football)" and that it gives me the answer that I put in the output. Showing the answer before and after the fine-tuning.

You could accept my request, because I don't have an NVIDIA card and I'm struggling 🥲.

My email : therealslayerinc@gmail.com

pereconteur commented 1 year ago

You would agree ? I would be very grateful and it would allow me to see how it works! @bofenghuang

bofenghuang commented 1 year ago

Hi @pereconteur!

Sure, I am happy to help you with your request!

However, I want to let you know that I am currently working on some new features that will affect the structure of this repository. It might be more beneficial for you to wait until these changes are implemented before proceeding with the video tutorial.

pereconteur commented 1 year ago

Hi @bofenghuang,

Okay. When do you think it will be ready? And can you make a step-by-step video with the example I gave for my viewers, so I can explain how to do it? They are very eager to know how to fine-tune!

pereconteur commented 1 year ago

Hi @bofenghuang,

I see you updated your repo! I'm very happy that you succeeded!

By the way, I would like to know if you could send me what I asked you? My subscribers are eager to know the steps to finish with your repo, thank you!

FxNion commented 1 year ago

Hi @bofenghuang

So much thanks for your job, il really believe that's a good way to play with french llm models.

I follow the wondefull instructions of @pereconteur to tranform vigogne model to llama.cpp models, in order to play with it on MacBook M1

It seems that "bofenghuang/vigogne-lora-7b" is no more available, as the command:

python export_state_dict_checkpoint.py 
             --base_model_name_or_path ../../llama-7B-hf 
             --lora_modele_name_or_path "bofenghuang/vigogne-lora-7b" 
             --output_dir ./models/7B --base_model_size "7B"

No more success with "bofenghuang/vigogne-7b-instruct"

python export_state_dict_checkpoint.py 
             --base_model_name_or_path ../../llama-7B-hf 
             --lora_modele_name_or_path "bofenghuang/vigogne-7b-instruct" 
             --output_dir ./models/7B --base_model_size "7B"

which both terminate by a sadly 

Could not consume arg: --lora_modele_name_or_path

I've read you where tranforming (^^) the repo, and may be there is no backward compatibility with the old tutorial ?

Thanks again

pereconteur commented 1 year ago

Hi @bofenghuang

So much thanks for your job, il really believe that's a good way to play with french llm models.

I follow the wondefull instructions of @pereconteur to tranform vigogne model to llama.cpp models, in order to play with it on MacBook M1

It seems that "bofenghuang/vigogne-lora-7b" is no more available, as the command:


python export_state_dict_checkpoint.py 

             --base_model_name_or_path ../../llama-7B-hf 

             --lora_modele_name_or_path "bofenghuang/vigogne-lora-7b" 

             --output_dir ./models/7B --base_model_size "7B"

No more success with "bofenghuang/vigogne-7b-instruct"


python export_state_dict_checkpoint.py 

             --base_model_name_or_path ../../llama-7B-hf 

             --lora_modele_name_or_path "bofenghuang/vigogne-7b-instruct" 

             --output_dir ./models/7B --base_model_size "7B"

which both terminate by a sadly 


Could not consume arg: --lora_modele_name_or_path

I've read you where tranforming (^^) the repo, and may be there is no backward compatibility with the old tutorial ?

Thanks again

Hi @FxNion !

I have a YouTube channel where I give lots of tutorials, including how to install Vigogne : https://youtu.be/BBf5h0HCFMY

Nevertheless I advise you first to join my Discord, where I update as much as possible all the new way to install, you'll just have to browse the different forum hello : https://discord.gg/xY63gyVfaR

I hope this helps and don't hesitate to give me your feedback!

FxNion commented 1 year ago

Salut @pereconteur ;)

C'est suite à ta super vidéo que j'ai atterri ici, pour tenter de résoudre les problèmes que j'ai rencontrés en tentant de marcher dans tes pas.

Et je suis aussi sur ton Discord, ou j'ai posé la même question.

Merci encore, et à bientôt !

pereconteur commented 1 year ago

Ok dans ce cas là > Salut @pereconteur ;)

C'est suite à ta super vidéo que j'ai atterri ici, pour tenter de résoudre les problèmes que j'ai rencontrés en tentant

  • [ ] de marcher dans tes pas.

Et je suis aussi sur ton Discord, ou j'ai posé la même question.

Merci encore, et à bientôt !

Merci beaucoup pour ton retour ! Dans ce cas là clique sur ce lien et si il ne marche pas, pose ta question dans la catégorie IA sans entraide : https://discord.com/channels/1092039071435599874/1113928879481032724

FxNion commented 1 year ago

Waouh @pereconteur, super boulot avec ce script. Je vais tester ça !

Merci pour ton aide

FxNion commented 1 year ago

@pereconteur a priori les urls des adapters_config.json ont changé, ça ne doit pas être grand-chose.

Entry Not Found for url: https://huggingface.co/bofenghuang/vigogne-7b-instruct/resolve/main/adapter_config.json

et 

Entry Not Found for url: https://huggingface.co/bofenghuang/vigogne-7b-chat/resolve/main/adapter_config.json.
bofenghuang commented 1 year ago

Hi @FxNion,

Thanks for your interest! The repository evolves rapidly, affecting backward compatibility. You could refer to the latest documentation here for reference.