Unable to train LoRA

[Cohejh]

Describe the bug

Whenever I try to train a LoRA, when it tries to reload the model, it breaks.

Is there an existing issue for this?

• [x] I have searched the existing issues

Reproduction

1. Load MythoMax-13b (GPTQ) (By TheBloke)
2. Go to Training
3. Fill in Parameters, then train.

Screenshot

No response

Logs

20:07:58-251522 INFO     Getting model ready...
Transformers Model Type: Exllamav2HF
20:07:58-252522 INFO     Preparing for training...
20:07:58-252522 INFO     Creating LoRA model...
20:09:09-136166 WARNING  LoRA training has only currently been validated for LLaMA, OPT, GPT-J, and GPT-NeoX models.
                         (Found model type: Exllamav2HF)
*** LoRA: LoRA_Test ***
20:09:14-137924 INFO     Loading JSON datasets...
Map: 100%|███████████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 384.57 examples/s]
BOS: True EOS: False
Data Blocks: 5
(Model has been modified by previous training, it needs to be reloaded...)
20:09:15-138891 INFO     Loading "None"
20:09:15-141889 ERROR    Failed to reload the model.
Traceback (most recent call last):
  File "C:\Users\jcohe\Documents\text-generation-webui\extensions\Training_PRO\script.py", line 774, in do_train
    reload_model()
  File "C:\Users\jcohe\Documents\text-generation-webui\modules\models.py", line 444, in reload_model
    shared.model, shared.tokenizer = load_model(shared.model_name)
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\jcohe\Documents\text-generation-webui\modules\models.py", line 87, in load_model
    output = load_func_map[loader](model_name)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\jcohe\Documents\text-generation-webui\modules\models.py", line 380, in ExLlamav2_HF_loader
    return Exllamav2HF.from_pretrained(model_name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\jcohe\Documents\text-generation-webui\modules\exllamav2_hf.py", line 168, in from_pretrained
    config.prepare()
  File "C:\Users\jcohe\Documents\text-generation-webui\installer_files\env\Lib\site-packages\exllamav2\config.py", line 70, in prepare
    assert os.path.exists(self.model_dir), "Can't find " + self.model_dir
AssertionError: Can't find models\None

System Info

Windows 11, Nvidia GeForce RTX 3060.

[Alekkc]

https://github.com/oobabooga/text-generation-webui/wiki Training is supported for Transformers and GPTQ-for-LLaMa loaders. It is not possible for your Exllamav2HF loader.

[Cohejh]

https://github.com/oobabooga/text-generation-webui/wiki

Training is supported for Transformers and GPTQ-for-LLaMa loaders. It is not possible for your Exllamav2HF loader.

Ok, thank you very much for your help

[github-actions[bot]]

This issue has been closed due to inactivity for 6 months. If you believe it is still relevant, please leave a comment below. You can tag a developer in your comment.