Using ComfyAction we can validate the performance impact of changes to ComfyStream or sample workflows
Motivation
It is important to validate performance impacts of changes to ComfyStream, we should research if a CI/CD workflow action can be improved to add additional metrics calculations. Currently it measures the time to run each workflow which would be a single frame. We also have FPS metrics available in JSON and prometheus format that can be used to measure FPS on a test stream
Technical Scope
Investigate two approaches:
Using the comfy-action github workflow, we want to explore if it is possible to automatically test our sample workflows in ComfyStream when PRs to main are opened. See Comfy-Action: https://github.com/Comfy-Org/comfy-action
A more simple approach might be to start ComfyUI/ComfyStream API server and send a test stream with http-trickle/ffmpeg
linear[bot]
commented
3 months ago
Feature type
CI/CD
Describe the feature
Using ComfyAction we can validate the performance impact of changes to ComfyStream or sample workflows
Motivation
It is important to validate performance impacts of changes to ComfyStream, we should research if a CI/CD workflow action can be improved to add additional metrics calculations. Currently it measures the time to run each workflow which would be a single frame. We also have FPS metrics available in JSON and prometheus format that can be used to measure FPS on a test stream
Technical Scope
Investigate two approaches:Additional context
No response