search

iiasa / GLOBIOM-G4M-link

Automatize the link between GLOBIOM and G4M
2 stars 1 forks source link

Create HTCondor submit container #17

Closed acjbrouwer closed 7 months ago

acjbrouwer commented 1 year ago

Create a container that provides as a service submission of Condor_run_R bundles to the Limpopo cluster. Notification and passing of input data and output data is TBD. Options:

When a shared volume is used, it is probably expedient to have one instance of this container per connection. When purely via a message broker, this can be a K8s-namespace-wide service.

acjbrouwer commented 11 months ago

Might be able to get away with instead instantiating interactive scientific stack containers that can submit from a co-installed HTCondor (see #18).

acjbrouwer commented 7 months ago

Set up automation for not just building a submit container but also an execution point container and a Central Manager container together with a pool bootstrap container. These are all built off the same base image. The base image is based on Ubuntu (22.04 currently) and pulls in HTCondor (23.04 currently) and additional packages, notably SupervisorD as an init system, and Git to pull in configuration.

The bootstrap container serves to generate the pool signing key and default token (for IDTOKENS authentication) provided to pods (container instances) via a Kuberenetes secret.