Added many upgrades for using models with CHEESE, as well as several other QOL changes. Of note is that CHEESE now supports offline evaluation (i.e. cases with no human in the loop). Some examples of where this would be useful could be to evaluate one model with another, or to score model outputs in some automated way. There's some kinks in how this works as a consequence of how rabbitmq works, but these are outlined in examples and documentation.
API:
Added option to use a progress bar with the API. This is pretty general but some examples for things it can be used for:
-> Checking how many tasks clients have completed
-> Checking how much data has been processed by model
Added stats as a dictionary accessible from the API, contains many useful numbers for the overall task and for the model/clients/pipeline specifically
Pipelines:
Did some more testing for saving progress
Added more extensive progress saving, with the intent of making it so nothing is lost/changes if the program crashes or is closed
Added a generative pipeline for model generated data. Can iterate over prompts and is batched. Runs continuously in a separate thread until a buffer is full to ensure there is always data to serve to clients.
Model:
Batched processing
Added a StableDiffusion example to demonstrate generative pipeline
Added many upgrades for using models with CHEESE, as well as several other QOL changes. Of note is that CHEESE now supports offline evaluation (i.e. cases with no human in the loop). Some examples of where this would be useful could be to evaluate one model with another, or to score model outputs in some automated way. There's some kinks in how this works as a consequence of how rabbitmq works, but these are outlined in examples and documentation.
API:
Pipelines:
Model:
Batched processing
Added a StableDiffusion example to demonstrate generative pipeline