search

Lightning-Universe / stable-diffusion-deploy

Learn to serve Stable Diffusion models on cloud infrastructure at scale. This Lightning App shows load-balancing, orchestrating, pre-provisioning, dynamic batching, GPU-inference, micro-services working together via the Lightning Apps framework.
https://lightning.ai/muse
Apache License 2.0
391 stars 39 forks source link

Issues in framework affecting autoscaling #95

Open rohitgr7 opened 2 years ago

rohitgr7 commented 2 years ago
  1. We need to register new works when traffic increases, but every time we spawn a new work, it takes a lot of time to provision, and most of the time is spent on installing the requirements even though it's the same in all the works. It should cache the requirements.
  2. Once work is stopped, it can't be restarted, so we used a workaround to use UUID once the app is running for too long, it creates a mess in the UI.
kaushikb11 commented 2 years ago

Also, not so pretty hack of using setattr for defining Works

rohitgr7 commented 2 years ago

yeah , there's a PR already on the platform side to resolve List and Dict structures.