GPT4All crashes when loading the model

nomic-ai / gpt4all

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

https://nomic.ai/gpt4all

MIT License

70.91k stars 7.72k forks source link

GPT4All crashes when loading the model #2767

Open osamaaz1 opened 4 months ago

osamaaz1 commented 4 months ago

The GPT4All program crashes every time I attempt to load a model. My laptop should have the necessary specs to handle the models, so I believe there might be a bug or compatibility issue.

Steps to Reproduce Open the GPT4All program. Attempt to load any model. Observe the application crashing.

Expected Behavior The model should load successfully without crashing the application.

Your Environment GPT4All version: 3.1.1 Operating System: Windows 11 22621.3880 (22H2) RAM: 16GB CPU: Intel Core i5 11400H GPU: NVIDIA RTX 3050ti Chat model used: Llama 3.1 8B Instruct 128k

armacoty commented 4 months ago

Same steps to reproduce.

Environment GPT4All version: 3.1.1 Operating System: Windows 11 Pro 22631.3958 (23H2) RAM: 48GB CPU: AMD Ryzen 7 5800X 8-Core Processor 3.80 GHz GPU: NVIDIA GeForce GTX 1050 Chat model used: Llama 3.1 8B Instruct 128k

cosmic-snow commented 4 months ago

Some things to check:

Does it work when you set Device to CPU in your settings?
Does it work when you set Device to Vulkan in your settings?
What did your set your Context Length to in your model settings?
What is your NVIDIA driver version?
What model were you using? Can you try again with a smaller one? For example, 'Phi-3 Mini Instruct' or 'Qwen2-1.5B-Instruct'

armacoty commented 4 months ago

It works when I set the CPU.
It not works when I set Device to Vulkan.
I am not changed it, it was 2048.
NVIDIA Driver: Game Ready 560.70 (Jul 16, 2024)
Smaller models work on CUDA

I think the problem is that memory shortage of the GPU is being handled incorrectly, which causes it to crash without any error message.

osamaaz1 commented 4 months ago

What is your NVIDIA driver version? 31.0.15.5244 ; 560.70 What did your set your Context Length to in your model settings? ### 2048 Does it work when you set Device to Vulkan in your settings? Does it work when you set Device to CPU in your settings? Issue resolved by setting GPU Layers = 1 What model were you using? Can you try again with a smaller one? Phi-3 Mini worked

cosmic-snow commented 4 months ago

Thanks for reporting back, and good that you got at least something working.

I think the problem is that memory shortage of the GPU is being handled incorrectly, which causes it to crash without any error message.

It should not crash in that case, but degrade gracefully.

You can also try playing around with the GPU Layers setting, by the way. Looks like that worked for osamaaz1.

yemreeee commented 3 months ago

I faced similar issue. Then I realized that when I decreased gpu layer rating it worked. (I'm working with 1050 Ti)

PhiBli commented 1 month ago

Mine crashed too...tested a bunch of versions. Switching to CPU under device did the trick. I use an ASUS laptop with a dedicated MX350. Thanks!