PelicanPlatform / pelican

The Pelican Platform for creating data federations
https://pelicanplatform.org/
Apache License 2.0
10 stars 19 forks source link

Idea: Have `pelican` invoke Docker #1501

Open bbockelm opened 1 month ago

bbockelm commented 1 month ago

Watching @aowen-uwmad's tutorial during HTC24, one idea came to me. Origins are currently started with the following command:

docker run --rm -it \
    -p 8444:8444 -p 8443:8443 \
    -v $(pwd)/config/issuer.jwk:/etc/pelican/issuer.jwk \
    -v $(pwd)/config/issuer-pub.jwks:/etc/pelican/issuer-pub.jwks \
    -v /etc/hostcert.pem:/etc/hostcert.pem \
    -v /etc/hostkey.pem:/etc/hostkey.pem \
    -v $(pwd)/config/pelican.yaml:/etc/pelican/pelican.yaml \
    -v $(pwd)/data:/data \
    hub.opensciencegrid.org/pelican_platform/origin:v7.9.2 \
    serve -p 8444

That's long and ugly enough that we ended up writing a small shell script (start-origin.sh in this repo) to avoid having users type this out.

Pelican, in principle, could parse its configuration file to determine the ports, the certificates, the data mounts, etc. We could have:

pelican origin serve --docker

be a shortcut for the above CLI invocation.

aowen-uwmad commented 1 month ago

Ideally the above Docker command would just mount /etc or /etc/pelican to shorten things up. But Pelican (or maybe more accurately XRootD) creates root-owned files/directories that cannot be removed by the user once the Docker container is exited. If things need to be cleaned up or reset, the user needs to either have sudo or (not tested) needs to run another Docker container with the mounts in order to remove those files.

I think this is a good idea, but will need to be careful about permissions for folks who have Docker access but not root access.

bbockelm commented 1 month ago

Yeah - I think that's surmountable. One could invoke the container as the current user, then bind mount all the mutable directories (run, logs) to corresponding directories inside the user's home.