search

oobabooga / text-generation-webui

A Gradio web UI for Large Language Models.
GNU Affero General Public License v3.0
39.67k stars 5.21k forks source link

Finetune Llama-2 error on encode call #3249

Closed LAXnACE closed 8 months ago

LAXnACE commented 1 year ago

Describe the bug

I'm trying to finetune llama2, but an error occurs when calling the encoder

Is there an existing issue for this?

Reproduction

Try to finetune llama2 model loaded via llama.cpp

Screenshot

No response

Logs

Traceback (most recent call last):
  File "C:\oobabooga_windows\installer_files\env\lib\site-packages\gradio\routes.py", line 427, in run_predict
    output = await app.get_blocks().process_api(
  File "C:\oobabooga_windows\installer_files\env\lib\site-packages\gradio\blocks.py", line 1323, in process_api
    result = await self.call_function(
  File "C:\oobabooga_windows\installer_files\env\lib\site-packages\gradio\blocks.py", line 1067, in call_function
    prediction = await utils.async_iteration(iterator)
  File "C:\oobabooga_windows\installer_files\env\lib\site-packages\gradio\utils.py", line 336, in async_iteration
    return await iterator.__anext__()
  File "C:\oobabooga_windows\installer_files\env\lib\site-packages\gradio\utils.py", line 329, in __anext__
    return await anyio.to_thread.run_sync(
  File "C:\oobabooga_windows\installer_files\env\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "C:\oobabooga_windows\installer_files\env\lib\site-packages\anyio\_backends\_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "C:\oobabooga_windows\installer_files\env\lib\site-packages\anyio\_backends\_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "C:\oobabooga_windows\installer_files\env\lib\site-packages\gradio\utils.py", line 312, in run_sync_iterator_async
    return next(iterator)
  File "C:\oobabooga_windows\text-generation-webui\modules\training.py", line 461, in do_train
    train_data = data['train'].map(generate_and_tokenize_prompt, new_fingerprint='%030x' % random.randrange(16**30))
  File "C:\oobabooga_windows\installer_files\env\lib\site-packages\datasets\arrow_dataset.py", line 580, in wrapper
    out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
  File "C:\oobabooga_windows\installer_files\env\lib\site-packages\datasets\arrow_dataset.py", line 545, in wrapper
    out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
  File "C:\oobabooga_windows\installer_files\env\lib\site-packages\datasets\arrow_dataset.py", line 3087, in map
    for rank, done, content in Dataset._map_single(**dataset_kwargs):
  File "C:\oobabooga_windows\installer_files\env\lib\site-packages\datasets\arrow_dataset.py", line 3441, in _map_single
    example = apply_function_on_filtered_inputs(example, i, offset=offset)
  File "C:\oobabooga_windows\installer_files\env\lib\site-packages\datasets\arrow_dataset.py", line 3344, in apply_function_on_filtered_inputs
    processed_inputs = function(*fn_args, *additional_args, **fn_kwargs)
  File "C:\oobabooga_windows\text-generation-webui\modules\training.py", line 457, in generate_and_tokenize_prompt
    return tokenize(prompt, add_eos_token)
  File "C:\oobabooga_windows\text-generation-webui\modules\training.py", line 341, in tokenize
    input_ids = encode(prompt, True)
  File "C:\oobabooga_windows\text-generation-webui\modules\training.py", line 329, in encode
    result = shared.tokenizer.encode(text, truncation=True, max_length=cutoff_len)
TypeError: LlamaCppModel.encode() got an unexpected keyword argument 'truncation'

System Info

CPU: AMD R9 5900X
RAM: 32GB
GPU: 3070Ti
Ph0rk0z commented 1 year ago

llama.cpp can't make lora unless something has changed recently.

LeoFitz87 commented 1 year ago

Having the same problem.

arthurwolf commented 1 year ago

Not the exact same error message, but a very similar one at the same line, see the issue I just opened.

arthurwolf commented 1 year ago

llama.cpp can't make lora unless something has changed recently.

so what should I use ??

steven4547466 commented 1 year ago

llama.cpp can't make lora unless something has changed recently.

so what should I use ??

Figure anything out?

kallebysantos commented 1 year ago

I almost made it 😢!!!!

llama.cpp can't make lora unless something has changed recently.

so what should I use ??

Try switch to LlamacppHF model loader, also was necessary download oobabooga/llama-tokenizer (described here #3499) Model used: TheBloke/Llama-2-7b-Chat-GGUF

logs:

File “/home/kallebysantos/projects/machine-learning/oobabooga_linux/text-generation-webui/modules/training.py”, line 505, in do_train

lora_model = get_peft_model(shared.model, config) File “/home/kallebysantos/projects/machine-learning/oobabooga_linux/installer_files/env/lib/python3.10/site-packages/peft/mapping.py”, line 106, in get_peft_model

return MODEL_TYPE_TO_PEFT_MODEL_MAPPING[peft_config.task_type](model, peft_config, adapter_name=adapter_name) File “/home/kallebysantos/projects/machine-learning/oobabooga_linux/installer_files/env/lib/python3.10/site-packages/peft/peft_model.py”, line 889, in init

super().init(model, peft_config, adapter_name) File “/home/kallebysantos/projects/machine-learning/oobabooga_linux/installer_files/env/lib/python3.10/site-packages/peft/peft_model.py”, line 111, in init

self.base_model = PEFT_TYPE_TO_MODEL_MAPPING[peft_config.peft_type]( File “/home/kallebysantos/projects/machine-learning/oobabooga_linux/installer_files/env/lib/python3.10/site-packages/peft/tuners/lora.py”, line 274, in init

super().init(model, config, adapter_name) File “/home/kallebysantos/projects/machine-learning/oobabooga_linux/installer_files/env/lib/python3.10/site-packages/peft/tuners/tuners_utils.py”, line 88, in init

self.inject_adapter(self.model, adapter_name) File “/home/kallebysantos/projects/machine-learning/oobabooga_linux/installer_files/env/lib/python3.10/site-packages/peft/tuners/tuners_utils.py”, line 222, in inject_adapter

raise ValueError( ValueError: Target modules [‘q_proj’, ‘v_proj’] not found in the base model. Please check the target modules and try again.

jellohouse commented 12 months ago

Any updates on this?

I'm trying to do the same also with TheBloke/Llama-2-7b-Chat-GGUF

BartAgterbosch commented 10 months ago

On normal training I'm getting TypeError: LlamaCppModel.encode() got an expected keyword argument 'truncation', regardless of the loader, then I read that someone had success with using Training_PRO, unfortunately that also throws me an error, which is TypeError: LlamaCppModel.decode() got an unexpected keyword argument 'skip_special_tokens'. If I switch to the llamacpp_HF loader and download the llama_tokenizer however (successfully) training will fail with the following error: AttributeError: 'NoneType' object has no attribute 'pad_token_id'

Anybody got any ideas?

nephi-dev commented 10 months ago

Same errors here, i don't think that it's possible to train a lora of a gguf model

alexsd06 commented 9 months ago

Same here. I made it to the error in the UI... What other models should we try to train a lora on if this is so bugged?

Ph0rk0z commented 9 months ago

You can train a HF format (bnb int8, int4,etc) model or gptq. Or train with llama.cpp itself.

HasaanIjaz commented 9 months ago

Training LoRAs, atm, is only supported for Model Loader is:

  1. Transformer
  2. GPTQ-for-LLaMa

To find the most recent information on this, visit the following official wiki page of the text-generation-webui repo: https://github.com/oobabooga/text-generation-webui/wiki .

github-actions[bot] commented 8 months ago

This issue has been closed due to inactivity for 6 weeks. If you believe it is still relevant, please leave a comment below. You can tag a developer in your comment.