search

buildstream-migration / bst-staging

GNU Lesser General Public License v2.1
0 stars 0 forks source link

Per-architecture CAS servers #709

Open Cynical-Optimist opened 3 years ago

Cynical-Optimist commented 3 years ago

See original issue on GitLab In GitLab by [Gitlab user @nanonyme] on Oct 11, 2018, 21:33

Buildstream should support having different CAS servers for different architectures. Discussed earlier with [Gitlab user @jjardon] that not all builder types are equal in eg freedesktop-sdk build system. You can apparently fairly easily have starvation with ARM builders. This results in some artifacts in CAS being more computationally expensive than others and should have higher likelihood of being preserved. As CAS itself should most likely not have the prioritization logic, an idea was raised of having multiple CAS servers/daemons behind hard disks so you would have separate buckets with separate quotas.

Cynical-Optimist commented 3 years ago

In GitLab by [Gitlab user @juergbi] on Oct 11, 2018, 22:54

Could we use the existing support for conditionals for this or do we need core support for this?

Cynical-Optimist commented 3 years ago

In GitLab by [Gitlab user @tristanvb] on Oct 12, 2018, 12:05

This sounds like a problem of scaling - I would be curious of what [Gitlab user @sstriker] might think about this as he usually has good insights when it comes to scaling problems.

I personally think the approach of having separate servers pushes the problem space onto the user to solve with configuration, for a problem which might get more complicated over time as their project scales and their configurations might have to change to adapt, and that doesn't sound very ideal.

Instead a dynamic solution would be a more attractive path. Perhaps multiple machines could back a single CAS server interface in some way, perhaps multiple daemons could be spawned on different machines when internet traffic becomes a bottleneck, maybe there are other solutions which do not push this problem into configuration space.

Cynical-Optimist commented 3 years ago

In GitLab by [Gitlab user @tristanvb] on Oct 12, 2018, 12:11

Could we use the existing support for conditionals for this or do we need core support for this?

I think this could at least be a decent workaround, although I'm not sure that we can support conditionals in user configuration (where users are allowed to override the artifact server on a per-project basis). Maybe option conditional resolution could be supported in the user configuration inside the project specific configurations ?