search

toriato / stable-diffusion-webui-wd14-tagger

Labeling extension for Automatic1111's Web UI
1.32k stars 229 forks source link

Allow model unloading from VRAM #31

Closed vibrantrida closed 1 year ago

vibrantrida commented 1 year ago

My gpu don't have enough VRAM, it would be nice to have a way to unload the tagger model from VRAM at will incase I run into "out of memory" error during inference

toriato commented 1 year ago

There is a bug in Keras where it is not possible to release a model from memory. Downgrading to keras==2.1.6 may fix the issue, but it may also cause compatibility problems with other packages. Using subprocess to create a new process may also solve the issue, but it can be too complex. For now, it seems that the best option is to keep the model in memory, as most users use the Waifu Diffusion model with onnx.

Related issue: https://github.com/keras-team/keras/issues/2102

I will open a new issue and leave it open until I find a way to unload the Keras model.