search

openSUSE / docserv

A server for building and publishing documentation with DAPS
https://opensuse.github.io/docserv/
GNU General Public License v3.0
4 stars 3 forks source link

build worker threads: pets v/ cattle #102

Open ghost opened 5 years ago

ghost commented 5 years ago

as a @aspiers remarked, our software follows a very pet-like model which does not really make sense given that we are trying to run it in a private cloud primarily. one negative outcome of this mismatch is that despite using the current biggest vm template of the ecp, we are hitting the ram ceiling & cpu ceiling of our machine pretty easily. one solution to that would be putting our worker threads in other vms somehow... is there any way to do this?

aspiers commented 5 years ago

Or if you had an OpenStack image you could simply boot it in ECP with a larger flavor :-)

svenseeberg commented 5 years ago

I already used the biggest template available in the ECP (like 4 or 8 cores and 16 GB memory). Can larger ones be created? I could not find an option.

aspiers commented 5 years ago

Oh wow, OK yeah - in that case it sounds like it would be worth splitting up the workers into separate VMs. Even though it is technically feasible for the ECP admins to add larger flavours, I'm not sure if it makes sense given the size of the individual compute nodes we have; that might risk unnecessary fragmentation of resources on the compute plane.

So I suggest starting a discussion thread on cloud@suse.de or https://chat.suse.de/channel/cloud about this - I'm sure you will get lots of great advice in either of those places. Another possibility might be to do it via a bunch of separate containers, although unfortunately the ECP doesn't support Magnum AFAICS.

svenseeberg commented 5 years ago

Okay. I think running the workloads on other hosts should be quite easy to achieve. The source and target dirs are already synchronized with rsync. And running daps2docker on another host via ssh shouldn't be a problem either.

However: I'm not sure if this is yet required. @sknorr has better insights if the build process is taking too much time on one VM.

What I'm saying is that it can be extended quite easily later on.