Can't load (almost) any model after latest update

[ZoroarkGER]

Describe the bug

The web UI was working fine for me, I could load 7b and 13b models without problems. But all of that changed after updating to the latest version, in which I now get a "Done! Press any key to continue..." every time I try to load a model. I have no clue why that is and why it's doing that but I hope this problem will be solved The only model that I seem to be able to run at the moment is the Guanaco 7b model, while all the other 13b models I have just won't load anymore.

I also don't seem to be the first and only person to encounter this problem and I've spent my whole day trying fixes that worked for others, which turned out not to work for me though.

Yes, I also tried a complete reinstallation of the web UI.

Is there an existing issue for this?

• [X] I have searched the existing issues

Reproduction

Try to load a 13b model in the latest branch of the web UI on a machine with similar specs to mine.

Screenshot

No response

Logs

bin D:\Downloads\oobabooga_windows\installer_files\env\lib\site-packages\bitsandbytes\libbitsandbytes_cuda117.dll
2023-06-17 18:24:23 INFO:Loading the extension "gallery"...
Running on local URL: http://0.0.0.0:7860

2023-06-17 18:44:50 INFO:Loading TheBloke_Wizard-Vicuna-13B-Uncensored-GPTQ...
2023-06-17 18:44:50 INFO:Found the following quantized model: models\TheBloke_Wizard-Vicuna-13B-Uncensored-GPTQ\Wizard-Vicuna-13B-Uncensored-GPTQ-4bit-128g.compat.no-act-order.safetensors
Loading model ...
Drücken Sie eine beliebige Taste . . .

System Info

My specs: 
- CPU: Intel Core i7 9700k 4.6Ghz 8 Core 16 Threads
- RAM: 2x 8GB DDR4 Ripjaws V 3200Mhz (Single-channel because MB broken :c)
- Graphics card: ROG Strix 2060 Super OC 8GB GDDR5
- PSU: 700W BeQuiet Pure Power 11 Non-modular
- MB: Gigabyte Z390D rev 1.0
- SSD on which Oobabooga web UI is running on: Kingston 512GB
- Case: Fractal Design Meshify C with Tempered glass (removed glass panel)
- CPU cooler: BeQuiet Pure Rock 2 with a Noctua NF-A12x25
- Fans: 1x Fractal Design 120mm in and 1x FD 120mm exhaust,
  and a BeQuiet Pure Wings 2 120mm on top drawing air inward.

[MillionthOdin16]

I think it's because of your choice of computer case and system fans 🙄

[Ph0rk0z]

To be fair, op probably had that pre written.

Perhaps try another loader like GPTQ-for-llama. It loads with autogptq now.

[ZoroarkGER]

To be fair, op probably had that pre written.

Perhaps try another loader like GPTQ-for-llama. It loads with autogptq now.

i tried all of the options, and i get the same thing over and over again, i've read that it may be a thing where it preloads the model into the system ram and then into the vram and that maybe my ram isnt enough but my ram maxes out at 10gb when loading any model and 7b models work as i already mentioned.

[Ph0rk0z]

Make a bigger swap file and try to load that way.

[ZoroarkGER]

Make a bigger swap file and try to load that way.

what's a swap file and how do i do that?

Edit: I knew what that was just under a different name

[m-from-space]

I think you are able to google what the swap file is for and how to set its size in Windows. In German it's called "Auslagerungsdatei".

[ZoroarkGER]

I think you are able to google what the swap file is for and how to set its size in Windows. In German it's called "Auslagerungsdatei".

Yea, that's exactly the word I knew it under. Though I already made a pretty big swap file for playing games like 7 Days to Die as it needs a lot of RAM, but I'll see if an even bigger one will do. I just wonder why it needs so much RAM out of a sudden or if it is even a RAM issue as it never occurred before.

[ZoroarkGER]

Thank you so much, I set a 32GB RAM swap file and it finally loaded, though with both my RAM and SSD maxed out at 100%.

[ZoroarkGER]

Make a bigger swap file and try to load that way.

For anyone who has the same issue as me, try this ^^^^