search

ParisNeo / lollms-webui

Lord of Large Language Models Web User Interface
https://lollms.com
Apache License 2.0
4.27k stars 537 forks source link

core dump when loading model #350

Open digitalw00t opened 1 year ago

digitalw00t commented 1 year ago

Expected Behavior

I downloaded the Wizard-Vicuna-7B-Uncensored.ggmlv3.q2_K., and clicking it in the model farm to load it.

Current Behavior

I click the model, and I get a core dump.

Steps to Reproduce

Please provide detailed steps to reproduce the issue.

  1. Download Wizard-Vicuna-7B-Uncensored.ggmlv3.q2_K
  2. Click the model to load it

Possible Solution

I have no idea why it's doing this.

Context

Running on a virtual machine on a Dell R930 with 24 core and 120gb of ram to run in CPU mode.

Screenshots

Binding c_transformers loaded successfully. No model selected Personality lollms mounted successfully but no model is selected Checking discussions database... ok Your personal data is stored here :/home/draeician/Documents/lollms Your personal data is stored here :/home/draeician/Documents/lollms debug mode:false Please open your browser and go to http://localhost:9600 to view the ui

digitalw00t commented 1 year ago

Found the issue. I created this vm on a proxmox server that was using the QEMU cpu type. I checked cat /proc/cpuinfo and noticed the cpu flags were REALLY limited. I powered down, switched to host, and now I can see the "avx2" in particular is showing up. Which I assume was the reason for the flag. Perhaps we can add a cpu flag check to the software to get ahead of this before a user has an issue? I mean so it can be detected that the cpu doesn't support or is too old to run in CPU mode only? I'll leave this issue up until I get a comment to see if this is something that might be needed.

ParisNeo commented 1 year ago

Hi, thanks for sharing. In fact, it depends on your binding. Some bindings support the abscence of the avx2 instructions and others don't. which binding are you using?