search

simonsobs-uk / data-centre

This tracks the issues in the baseline design of the SO:UK Data Centre at Blackett
https://souk-data-centre.readthedocs.io
BSD 3-Clause "New" or "Revised" License
2 stars 1 forks source link

Continuous Deployment (CD) to CVMFS #26

Closed ickc closed 4 months ago

ickc commented 8 months ago

Currently, accessing the CVMFS publishing node requires one to open a link in the browser to authenticate first.

How to achieve CD then? Is there any option to authenticate without human interaction?

ControlMaster in ssh config can be useful but it requires running it on a instance that are persistent (such as our workstation), but not GitHub Actions.

ickc commented 8 months ago

Hi, @rwf14f, it is probably a question for you. Now that I can publish to CVMFS, what is the recommended strategy to continuously deploy to CVMFS? For example, the way I upload it right now requires me to ssh into it, which asks me to authenticate via an URL. How would automatic deployment be setup? Would it involve a host certificate?

Thanks.

afortiorama commented 8 months ago

You can only (gsi)ssh to upload the software. It is not foreseen to use host certificates for this, but you can do it with your x509 proxy if we add you to the lcgadmin role (this at least doesn't require a URL). However it will not be "continuous". How often do you plan to update the software? CVMFS is not really for software development you have a lag of more than 1h between uploading and the software appearing.

ickc commented 8 months ago

@afortiorama, thanks. “Continuous” as of right now is daily (nightly) build (and I personally don’t foresee us doing higher frequency than that). I already setup up a GitHub Actions to generates the artifacts. The remaining piece is to unarchive the tarball at the publishing node.

It would be great if there’s a way to let the GitHub Actions to deploy there directly, but I expect the grid infrastructure probably is not very suitable for this. So if there’s a cron job I can setup on the publishing node that accomplish this (running a script that download and unarchive), it is a good alternative. Or if I need to setup the cron on another machine which basically uses an AC cert then it is fine too.

So I guess the point is to automate it so that I don’t have to run that script manually on a daily basis. Generating AC once every week could be ok, although in holidays or annual leave would becomes challenging.

ickc commented 4 months ago

Marked as wontfix as it seems the infrastructure for doing this does not exist.