make "freeze_version" freeze them in the "super-dataset" instead of the ReproNim/containers

[yarikoptic]

as we realized/documented while going through https://github.com/ReproNim/containers#walkthrough with @asmacdo recently, freeze_versions modifies this repository/dataset locally and thus require resharing for that modification. That is inconvenient. Instead, in cases where used YODA way, thus being embedded into super-dataset for results, I think we can "freeze" via copying that container definition into super-dataset and just adjusting the path to the versioned image to reside in the sub-dataset! Then users can easily upgrade containers subdataset, while still using the container of the original version since that is what they have in superdataset.

Instead of freeze_version I guess we could formalize it as a new helper, e.g. use_container. It would imply use in superdataset, but I don't want to make the name too verbose/long.

[yarikoptic]

ha -- script already has an option --save-dataset= which is barely used to provide dataset path for datalad save which seems to be a bit detached from the rest of the logic. I will RF so whenever we point with that option to some other dataset above -- that is where to we copy the invocation section while adjusting the paths etc.

[asmacdo]

I still like it, using that should be part of the standard workflow.

maybe s/freeze/install/. ?

[yarikoptic]

I thought about install but disliked it since it doesn't really copy the container image, just merely "registers" it in .datalad/config file.

[asmacdo]

why not also copy the container?

maybe the containers dataset should only be what they use, rather than all ReproNim containers.

On Fri, Mar 3, 2023, 3:57 PM Yaroslav Halchenko @.***> wrote:

I thought about install but disliked it since it doesn't really copy the container image, just merely "registers" it in .datalad/config file.

— Reply to this email directly, view it on GitHub https://github.com/ReproNim/containers/issues/86#issuecomment-1454120182, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAH3EMIPGKIF6XBCQVVQTBLW2JLNHANCNFSM6AAAAAAVPAGJNM . You are receiving this because you were mentioned.Message ID: @.***>

[yarikoptic]

why not also copy the container?

well, in principle indeed could have a command for a full copy of the container, which would need to use datalad copy-file.

pros:

• only container(s) used are contained
• no need to install heavier ReproNim/containers

cons:

• would need copy also of the shim script and ensuring the possibly needed similar setup for isolated HOME, licenses/ etc. Sounds like a burden not worth taking
• if container image availability information updated in ReproNim/containers -- it will not be anyhow sync'ed back to the superdataset
• if shim is copied, then also would need to be similarishly copied on upgrades

overall -- feasible but probably not worth pursuing because of the cons ATM

maybe the containers dataset should only be what they use, rather than all ReproNim containers.

we do not restrict people to create their own container datasets. people could even trim this dataset with git rm to what they want ... the point is that they can use this one and get benefits (big collection at fingertips, shim to ensure various aspects, later metadata to discover).

[yarikoptic]

It was addressed by #88