can't load models

[MenrichKoller]

Describe the bug

since last week's update, I can't load any gguf model. i can still do so if i roll back.

Is there an existing issue for this?

• [X] I have searched the existing issues

Reproduction

try to load a gguf model while running cpu only on windows

Screenshot

No response

Logs

Traceback (most recent call last):

File "C:\Users\Menrich\Documents\ai-stuff-3-gpt-stuff-like-automatic1111\text-generation-webui\modules\ui_model_menu.py", line 248, in load_model_wrapper

shared.model, shared.tokenizer = load_model(selected_model, loader)

                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Menrich\Documents\ai-stuff-3-gpt-stuff-like-automatic1111\text-generation-webui\modules\models.py", line 94, in load_model

output = load_func_map[loader](model_name)

         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Menrich\Documents\ai-stuff-3-gpt-stuff-like-automatic1111\text-generation-webui\modules\models.py", line 271, in llamacpp_loader

model, tokenizer = LlamaCppModel.from_pretrained(model_file)

                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Menrich\Documents\ai-stuff-3-gpt-stuff-like-automatic1111\text-generation-webui\modules\llamacpp_model.py", line 63, in from_pretrained

Llama = llama_cpp_lib().Llama

        ^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'Llama'

System Info

running on win 10,
running cpu only,
intel i9 10900k,
rx 6800xt (though i am running cpu only)

[geeky-devil]

I had same issue, for some reason the imports were not working properly. running on terminal instead on an IDE fixed the issue for me. hope it helps you

[MenrichKoller]

I had same issue, for some reason the imports were not working properly. running on terminal instead on an IDE fixed the issue for me. hope it helps you

I am not sure what you mean. I for one always run it by clicking the start_windows.bat file. how would i run it with a terminal, or am I doing that by using that bat file?

[Sobhanysiamak]

ممنون از راهنمایی شما

[MenrichKoller]

ممنون از راهنمایی شما

if this really does translate to "thank you for your guidance" as google translate says then: what? what guidance did I give?

[geeky-devil]

I had same issue, for some reason the imports were not working properly. running on terminal instead on an IDE fixed the issue for me. hope it helps you

I am not sure what you mean. I for one always run it by clicking the start_windows.bat file. how would i run it with a terminal, or am I doing that by using that bat file?

at first when u run the bat file , it installs the conda ,python and the required packages. you can actually skip using the bat file and directly run the python file (server.py) with custom args. example: python server.py --listen-port 8080 check the README , everything has been explained there.

[MenrichKoller]

I had same issue, for some reason the imports were not working properly. running on terminal instead on an IDE fixed the issue for me. hope it helps you

I am not sure what you mean. I for one always run it by clicking the start_windows.bat file. how would i run it with a terminal, or am I doing that by using that bat file?

at first when u run the bat file , it installs the conda ,python and the required packages. you can actually skip using the bat file and directly run the python file (server.py) with custom args. example: python server.py --listen-port 8080 check the README , everything has been explained there.

thank you, but I can't figure it out. If I keep trying, I'm sure I'd eventually figure it out, but I'll just stick to the slightly older version until the issue gets fixed.

[geeky-devil]

UPDATE I ran into same issue again when using the latest version. the issue was with the import so i tried debugging.

using this i got the error msg as

so i am not using the bat file and mini-conda to run the ui cause i already had python3.10 i checked the start_windows.bat and saw that the CUDA_HOME and CUDA_PATH were set to conda env. i set those to my python env D:/python3.10

as you can see it says it cant find the bin folder , so i created it and it solved the issue. apparently the conda-env has bin folder in it.