d8ahazard / sd_dreambooth_extension

Other
1.86k stars 282 forks source link

"FATAL: this function is for sm80, but was built for sm750" #74

Closed yami-no-tusbas closed 1 year ago

yami-no-tusbas commented 1 year ago

I dunno what's going on, it seems to train, but... It show this error a lot, a very huge lot... Like every 0.2 seconds a new line of this FATAL: this function is for sm80, but was built for sm750 pop in my console.

I'm on windows, using automatic1111 GUI's with this plugin. I have a NVIDIA RTX4090 24Gb VRAM.

d8ahazard commented 1 year ago

Did you restart the webUI entirely after installing?

yami-no-tusbas commented 1 year ago

Ok to be sure I shut it off, the relaunched it, But yeah, I installed the plugin like this morning, but didn't use it because it was a bit confusing to me, since a tutorial was done on youtube I gave it a try.

Here is the starting logs :

venv "D:\stable-diffusion-webui\venv\Scripts\Python.exe"
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Commit hash: ac085628540d0ec6a988fad93f5b8f2154209571
Installing requirements for Web UI
Installing Requirements for Dreambooth
Diffusers version is 0.8.0.dev0.
Copying 8Bit Adam files for Windows.

Launching Web UI with arguments: --deepdanbooru
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
making attention of type 'vanilla' with 512 in_channels
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
making attention of type 'vanilla' with 512 in_channels
Loading weights [a2a802b2] from D:\stable-diffusion-webui\models\Stable-diffusion\luxddim_2.ckpt
Applying cross attention optimization (Doggettx).
Model loaded.
Loaded a total of 24 textual inversion embeddings.
Embeddings: ...

To create a public link, set `share=True` in `launch()`.

Ok there was an update for your plugin, it's done.

And there it is :

 VRAM cleared.
 Allocated: 0.0GB
 Reserved: 0.0GB

Trying to parse: [{'instance_prompt': 'portrait of luxddim person', 'class_prompt': 'photo of a person', 'instance_data_dir': 'D:\\training\\Lux_b\\focal_crop', 'class_data_dir': 'D:\\training\\regularisation\\person'}]
Unable to load concepts as JSON, trying as file: Expecting property name enclosed in double quotes: line 1 column 3 (char 2)
Loaded concepts from [{'instance_prompt': 'portrait of luxddim person', 'class_prompt': 'photo of a person', 'instance_data_dir': 'D:\\training\\Lux_b\\focal_crop', 'class_data_dir': 'D:\\training\\regularisation\\person'}]
 Loaded model.
 Allocated: 0.0GB
 Reserved: 0.0GB

 Scheduler Loaded
 Allocated: 0.2GB
 Reserved: 0.2GB

Steps:   0%|                                                                                  | 0/1200 [00:00<?, ?it/s]FATAL: this function is for sm80, but was built for sm750
FATAL: this function is for sm80, but was built for sm750

My GUI is up to date, the plugin too,

Here is nvidia-smi info too just in case :

+-----------------------------------------------------------------------------+ PS C:\Users*> nvidia-smi Wed Nov 9 20:04:35 2022 +-----------------------------------------------------------------------------+ | NVIDIA-SMI 526.47 Driver Version: 526.47 CUDA Version: 12.0 | |-------------------------------+----------------------+----------------------+ | GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 NVIDIA GeForce ... WDDM | 00000000:09:00.0 On | Off | | 0% 54C P8 37W / 450W | 19859MiB / 24564MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+ +-----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| | 0 N/A N/A 3636 C+G ... iCUE 4 Software\iCUE.exe N/A | | 0 N/A N/A 5660 C+G ...8bbwe\WindowsTerminal.exe N/A | | 0 N/A N/A 7736 C+G C:\Windows\explorer.exe N/A | | 0 N/A N/A 8264 C+G ...perience\NVIDIA Share.exe N/A | | 0 N/A N/A 9496 C+G ...n1h2txyewy\SearchHost.exe N/A | | 0 N/A N/A 9504 C+G ...artMenuExperienceHost.exe N/A | | 0 N/A N/A 11736 C+G ...\app-1.0.9007\Discord.exe N/A | | 0 N/A N/A 16112 C+G ...6.0.3.0\GoogleDriveFS.exe N/A | | 0 N/A N/A 16444 C+G ...ge\Application\msedge.exe N/A | | 0 N/A N/A 17652 C+G ...y\ShellExperienceHost.exe N/A | | 0 N/A N/A 18228 C D:\Python_3.10.6\python.exe N/A | +-----------------------------------------------------------------------------+

d8ahazard commented 1 year ago

FATAL: this function is for sm80, but was built for sm750

Are you using xformers? I only found one other mention of this issue anywhere else, and it seemed to possibly be related to needing to compile xformers for your GPU.

yami-no-tusbas commented 1 year ago

Hmm, at the beginning I compiled xformers and all, but nowadays it's not used anymore i thing, so i did uninstall everything nvidia xformers compilation related and I removed --xformers from the bat file, but I think maybe this as something to do with xformers, but I have no idea how to reset that thing.

I found what I did at the time : https://rentry.org/sdg_faq#xformers-increase-your-its-more-cards-supported

yami-no-tusbas commented 1 year ago

Ok solved by deleting the xformers folder from the venv then launching it with --xformers to have it download them again, mine where "old" ones I compiled with the wrong version of the architecture (at the time I did not know that 4000 series are 8.9 and I used 8.6 (rtx3000 series) wich could explain the problem), now I got no error anymore.

alfredcs commented 1 year ago

Rebuilt xformer solved my same problem.

# Makes the build much faster
pip install ninja
# Set TORCH_CUDA_ARCH_LIST if running and building on different GPU types
pip install -v -U git+https://github.com/facebookresearch/xformers.git@main#egg=xformers
mavisyyc commented 1 year ago

Did you restart the webUI entirely after installing?

what will happen if i forgot to shut off the former webUI and relunched it