search

shauray8 / continuity

contains all the evals not available at lm-eval-harness and layer wise inference for bigger models with GGUF support
MIT License
2 stars 1 forks source link

Writing the step function for diffusion engine #2

Open shauray8 opened 1 month ago

shauray8 commented 1 month ago

Keeping it similar to vllm (at least for now), should basically have 

Start Step
    |
Check Parallel Config
    |
Retrieve Cached Outputs
    |
Clear Outputs
    |
Check Remaining Steps
    |
    +-----------------------------+
    |                             |
   Yes                           No
    |                             |
Schedule Next Iteration       Process Final Outputs
    |                             |
Process Outputs                   |
    |                             |
Check for Scheduled               |
Outputs                       Return Outputs
    |                              |
    +-----------------------------+
    |                              |
   Yes                             No
    |                              |
Construct Execute Model        Process Final Outputs
Request
    |
Execute Model
    |
Update Cached Outputs
    |
Append Outputs
    |
Process Final Outputs
    |
Return Outputs

probably will change a lot of this as per the needs and things I find fit for diffusion pipeline, for now this is what I'm following atleast for the step() function