search

shauray8 / continuity

contains all the evals not available at lm-eval-harness and layer wise inference for bigger models with GGUF support
MIT License
2 stars 1 forks source link

Entry point class before everything else #4

Open shauray8 opened 1 month ago

shauray8 commented 1 month ago

Writing an entry point class which essentially is just -

A diffusion module for generating images from given prompts and sampling parameters.

This class will includes a tokenizer, a diffusion model (possibly distributed across multiple GPUs) (not sure about that), and GPU memory space allocated for intermediate states (aka KV cache). Given a batch of prompts and sampling parameters, this class generates images from the model, using an intelligent batching mechanism and efficient memory management.

At least this is the goal, kinda transferring everything from vllm to diffusion

shauray8 commented 2 weeks ago

Entrypoint is kinda done until we get DiffusionArgParse and DiffusionEngine up and running