As requested by @db0 as a result of @Unit1208's recent attention (which we are very thankful for) to the image model reference repository, more models have been added at an increased rate than historically.
The problems
Workers may select commands like TOP 10 and never offer models new to the horde or outside of a very limited pool of already established models.
Workers may select commands like ALL, download 600+gb, only to find that within a few days or weeks, they now have to download 40gb upon starting their worker up, and either not have the budgeted space or may not be willing to dedicate hours to downloading (which also relates to a long standing issue).
Users continue to use popular models more often due to better response times and worker availability
These issues potentially cause the following issues that need addressing:
Established models with wide appeal are offered more often than new-to-the-horde models, with causes on both the user-side and worker-side.
New models are often not available shortly after being officially allowed
Worker operators who deliberately offer few models have to stay apprised of image model reference changes if they want to offer new models.
Worker operators may suddenly need to download many gigabytes of data, which they either may not have room for, may not have budgeted the time for, or may simply want to forgo at the moment.
The workflow for release should be adjusted to either generate a new json file of the models in stable_diffusion.json with the following information:
date_added
date_removed
date_modified
New model meta load instructions
ALL NEW
# NEW / NEW #
NEW SDXL #
Possible new other configuration options
avoid_new_models / avoid_new_models_day_threshold
For workers who use model meta load commands such as ALL or TOP, this being on should prevent a model within the avoid_new_models_day_threshold from being downloaded/offered
Models skipped over in this way should be included as a warning message to the operator at start and while the worker is running to give them a heads up of the incoming download and the download sizes.
This will give worker operators more control over when to opt-in to a download.
freeze_models_to_load
If true, load the latest resolved model list when the worker was actually running (after resolving all meta load commands), ignoring what the current state would be.
This would require the worker to store the last loaded models list somewhere
This hopefully should allow workers who find that the model load resolution has changed but do not currently want to download them and want to just run the latest stable config
It is worth pointing out that certain worker operators have explicitly requested some or all of these features.
As requested by @db0 as a result of @Unit1208's recent attention (which we are very thankful for) to the image model reference repository, more models have been added at an increased rate than historically.
The problems
TOP 10and never offer models new to the horde or outside of a very limited pool of already established models.ALL, download 600+gb, only to find that within a few days or weeks, they now have to download 40gb upon starting their worker up, and either not have the budgeted space or may not be willing to dedicate hours to downloading (which also relates to a long standing issue).These issues potentially cause the following issues that need addressing:
Proposed solution
Therefore, in response, I propose the following:
stable_diffusion.jsonwith the following information:date_addeddate_removeddate_modifiedALL NEW# NEW/NEW #NEW SDXL #avoid_new_models/avoid_new_models_day_thresholdALLorTOP, this being on should prevent a model within theavoid_new_models_day_thresholdfrom being downloaded/offeredfreeze_models_to_loadIt is worth pointing out that certain worker operators have explicitly requested some or all of these features.