Feature: Ray Alternative (Instant Model Loading)

[jug-dev]

Adds nataili.enable_ray_alternative switch which, if enabled (disabled by default) uses an alternative to Ray.

This feature changes nataili to use a persistent disk cache, rather than the ray object store which is lost each time the worker is restarted.

Benefits:

1. Worker starts instantly, no long period loading models into Ray. (After the first time it's loaded)
2. Testers have reported the worker being more stable and using less RAM.

Downsides:

1. Ray caches models in RAM and well as on disk. This Ray alternative does not use RAM it only uses disk. Therefore, if the number of models loaded will fit in RAM, the worker should continue to use Ray for better performance.

Note that Ray is still used, just not used for caching the large ckpt models.

The model cache files are stored in the folder model-cache in the Ray temp dir, if the ray temp dir has been moved using it's environmental variable (RAY_TEMP_DIR), the model cache files will also move with it.

Instructions For Testing

Change AI-Horde-Worker/requirements.txt contents to be:

--extra-index-url https://download.pytorch.org/whl/cu117
git+https://github.com/jug-dev/nataili.git@feature-ray-alternative#egg=nataili
xformers
gradio

Run ./update-runtime.cmd (or .sh on Linux)

Edit the file AI-Horde-Worker/worker/argparser/stable_diffusion.py to add "enable_ray_alternative" to the imports on line2:

from nataili import disable_progress, disable_voodoo, disable_xformers, enable_local_ray_temp, enable_ray_alternative

and add the following line at the end of the file:

enable_ray_alternative.activate()

[jug-dev]

An empty merge request, left alone, Forgotten by those who once did own, The code it held now left to roam, In limbo it rests, with no home.

No comments, no reviews, no merge, Just silence now, an eerie dirge, The author waits, with bated breath, For any sign, of life or death.

But days turn to weeks, and weeks to months, And hope begins to fade and confronts, The sad reality that this request, Will forever remain unaddressed.

So let this be a warning to all, Don't leave your merge requests to fall, Into the abyss of neglect and decay, Give them attention, every day.