Explore solutions for facilitating higher resolution generation

[dmarx]

• patch generation + stitching
  • might actually work well in VQGAN mode where we could segment patch in the latent space
• patch generation + stitching + seam repair
  • add (optional?) patch synthesis post-processing step to blend patch seams
  • there's definitely research here, don't need to come up with this out of thin air
• gradient checkpointing
  • assumes gradient VRAM is the main bottleneck here, which it may not necessarily be
• recommended settings
  • a la turbo mode, demonstrate some set of hyperparameters that makes high resolution generation feasible in current paradigm subject to various tradeoffs
• super-resolution as a post-processing step
  • may need to add some customization for inter-frame coherence or find a model specifically for video SR
• only portions of image updated per step
  • high cutout power, low number of cutouts
  • scale up steps_per_frame to compensate.
  • may also need to mask out non-cutout regions of image? double check how gradient is computed

[dmarx]

Per rAInisto:

fullHD 1920x1080 would be nice. With A6000 48GB and video source I can't quite reach even that right now.

TO DO: confirm this is not already achievable using gradient_accumulation. If it is, prioritize documenting this with a pytti-book tutorial.

[dmarx]

not necessarily for this issue specifically, but some general inspiration re performance optimization: https://horace.io/brrr_intro.html