search

ComputeCanada / puppet-magic_castle

Puppet Environment repo for Magic Castle - https://github.com/ComputeCanada/magic_castle
MIT License
13 stars 21 forks source link

Warm up CVMFS cache #101

Open ocaisa opened 3 years ago

ocaisa commented 3 years ago

It would be nice if it were possible to warm up the CVMFS cache, particularly for JupyterHub related modules. You could probably do this by leveraging EBROOT* from your environment when you have the modules loaded to construct a CVMFS dirtab file. I guess this should be pulled in by the squid proxy.

mboisson commented 1 year ago

With the “naive” approach, you could do for d in $(env | grep EBROOT); do ls ${d##*=}; done after loading the modules. ls will list the directories and fetch the root catalogs.

In most cases, we expect have one catalog per EBROOT<name>, value, at least with the Compute Canada software stack, not sure with EESSI, so that may be dependent on the software stack that is being used.

That would not fetch all files though. To fetch all files, I guess you could do for d in $(env | grep EBROOT); do find ${d##*=} -print0 | xargs -0 touch &>/dev/null; done

Maybe there is a better cvmfs command to do that however.

mboisson commented 1 year ago

From internal discussion, touch does not do it, one must read the files