search

VIDA-NYU / reproserver

A web application reproducing ReproZip packages in the cloud.
https://server.reprozip.org/
BSD 3-Clause "New" or "Revised" License
10 stars 8 forks source link

How to scale ReproServer? #46

Open remram44 opened 3 years ago

remram44 commented 3 years ago

Originally opened 2019-11-14 22:36 EST by @remram44

Not a problem for now, but worth thinking about.

Running multiple web pods wouldn't work because specific ones would know about specific builds/runs.

Changing VM states to not trigger multiple builds might be good but still cause issues. Plus container restart would have all reproservers watching all running pods.

I think the best option is probably to keep one reproserver per cluster, and just run multiple clusters. This is also an opportunity for federation, e.g. there could be "nyu reproserver" and "umass reproserver" and links would point to one specific server and everything would work. It could also be used to have institutional instances providing better quotas to their own employees.

remram44 commented 3 years ago

Originally posted by @nust

Have you considered building ReproServer on top of Binder? I guess you're much more flexible now in what infrastructure you can provide, but the federation aspect would come with the BinderHubs.

Originally posted by @remram44

I think running Binder on top of ReproServer might make more sense? But really they are very similar systems.