Haidra-Org / horde-worker-reGen

The default client software to create images for the AI-Horde
https://aihorde.net/
GNU Affero General Public License v3.0
94 stars 42 forks source link

Maintenance mode should free resources #264

Open tazlin opened 3 months ago

tazlin commented 3 months ago

There are a couple of possible approaches to this:

  1. The worker simply instructs all ComfyUI processes to unload models using the usual mechanisms (UNLOAD_MODELS_FROM_RAM message).
    • This has the side effect of retaining ~200mb of RAM per process
    • The safety process does not currently have an unload mechanism. Either one would be need to be implemented, or the safety process could alternatively be stopped - which would require the worker not attempt to automatically restart it as is currently the case.
  2. The worker shuts down all sub-processes, including the safety process.
    • It may be sufficient to shut all but one inference process and unload it's models, which would reduce ramp up time once maintenance mode is cleared.
    • Again, the worker would need a flag to avoid reloading these halted processes once they are halted for maintenance.