acjbrouwer
commented
10 months ago Preliminary choices
- Deploy central manager (collector and negotiator daemons) on Kubernetes.
- Use HTCondor 23.0.x Long Term Support series.
- Use token-based authentication.
- Not documented well, but Alexey managed to get it going on official htcondor containers under a Docker-on-Linux runtime. After excluding other auth methods from the configuration also works together with a Windows desktop submit node.
Targeted deployments
For reference see documentation section The Different Roles a Machine Can Play
- Central manager
- Container on K8s.
- With a logging sidecar container? (should be possible)
- With a status page webserver? (probably, Alexey has something like this going for the existing pool)
- Execute Points
- Windows Sever machines (limpopo1-6)
- Windows Server VMs (limpopo7-8)
- Windows desktops as execute points still? (probably not, no ECC, no controlled stack, and very little incremental capability relative to the new hardware)
- Arbitrary stack Linux containers on K8s (limpopo7-8 and other machines).
- Access Points
- Windows desktops
- Todo: refresh
I:\Smirnov\CONDOR_SETUPto HTCondor 23.0.x with updated configuration. - Container with a scientific stack (trialing installing HTCondor 23.0.x on an Ubuntu stack which gives access to many scientific stacks).
- Submit-as-a-service container taking in snapshot bundles? (may not be necessary if the former point works out well)
Action points
Alexey:
- Create new setup installer+config+scripting for a Windows Access Point deployment:
- With Condor 23.0.latest
- With some README documentation on how to install and get going (e,g, request a token).
Alexey & Albert:
- Managed HTCondor configuration for the various deployment kinds.
- Supporting IDTOKENS
- Supporting remote HTCondor administration
- For now not supporting RUN_AS_OWNER
Albert:
- Build custom containers for:
- Central Manager
- Linux Execute Point with scientific compute stack
- Linux Access Point with interactive scientific compute stack.
- Reproducible Kubernetes orchestration for deploying, redeploying, upgrading the above.
Limpopo still mostly runs HTCondor 8.x. Base containers for creating a submit container are available for HTCondor 10.x. Connecting to Limpopo with an instance thereof does not work. The Limpopo HTCondor deployment needs to be upgraded.