Voila at scale - Githubissues

voila-dashboards / voila

Voilà turns Jupyter notebooks into standalone web applications

https://voila.readthedocs.io

Other

5.4k stars 501 forks source link

Voila at scale #1195

Open alexisjons opened 2 years ago

alexisjons commented 2 years ago

Does voila scale up to support 500+ users on 3-6 servers? Or is it more of an individual user application for rendering jupyter notebooks?

timkpaine commented 2 years ago

This depends on the contents of the notebook itself, I have successfully deployed voila-based applications to hundreds of users when sharded into several instances and only interaction-based work occurring in the notebook. If you have heavy workloads in your notebooks, or heavy background processes, you're going to be limited in your scalability.

alexisjons commented 2 years ago

Thanks! How did you do it? Does load balancing work? Or did each server have it's own url?

Nephdz commented 1 year ago

@alexisjons Did you have any luck attempting to scale your applications? I'm trying to increase the number of instances the app is running on but I keep getting 404s.

jgunstone commented 1 year ago

to my knowledge the out-the-box config of voila is 1server / 1user which doesn't scale that well to 100's of users. there are ways customise the config and your app to make it work, others know more -

active developments happening:

solara is tackling this problem by having multiple users per server making it more scalable
voici uses jupyterlite so the app is served by the browser, making it infinitely scalable... but unless the data for the app is public you'll need a way of connecting to a private resource

edublancas commented 1 year ago

My team is working on a platform to (among other things) scale Voila applications. A few thoughts:

The main challenge when scaling Voila apps is that each app requires an IPython kernel (i.e. a Python process); this makes Voila a lot more resource-intensive than typical web apps (e.g. flask, fastAPI).

One thing you can do to alleviate this is to shut down idle kernels as soon as the user disconnects (Voila has options to do this), you can also use a load balancer to spin up new instances when scale increases.

Also, if you're doing health checks on your services, you can hit a static file in the Voila app, this will prevent kernel creation on each health check.

Happy to share more details, feel free to contact me.

trungleduc commented 1 year ago

The main challenge when scaling Voila apps is that each app requires an IPython kernel (i.e. a Python process); this makes Voila a lot more resource-intensive than typical web apps (e.g. flask, fastAPI).

Indeed this is the point we want to improve in the future. On the other hand, using a kernel allows Voila to handle properly the resource-intensive task, which is complicated in typical web apps.

timkpaine commented 5 months ago

Thanks! How did you do it? Does load balancing work? Or did each server have it's own url?

load balancer in front, sharded to maybe 5-10 instances. Preheated kernels is helpful as well, keeping whatever is done via interactivity to have minimal load was also important. Then heavy compute was also farmed out to a cluster.