Snap plugins contain other copy of Certbot

[bmw]

I may be mistaken, but I believe each separately packaged snap plugin includes its own copy of Certbot and all of its dependencies. Is there anyway we can avoid this by having the plugin essentially depend on the Certbot snap, pull it into its build environment, and then only install dependencies that aren't already satisfied?

[basak]

Good idea!

I think this would work in the case of most plugins. They could ship only the CERTBOT_PLUGIN_PATH directory and any dependency directories and that would make the plugin snaps much smaller.

If the list of directories is available, then I believe this is trivial to configure in snapcraft.yaml. I'm not sure how to get that list automatically though.

So I think our architecture allows it, but I'm not sure how in the build step to ensure that it is done correctly in the general case. In specific cases it should be relatively easy to arrange by considering each plugin individually.

[bmw]

What cases do you have in mind where you think this would not work?

To set this up, are you imagining the plugin including a subset of the directories under $SNAP/lib/python3.5/site-packages?

Is there any (easy) way to load the Certbot snap or its files into the build environment of the plugin snap? The Certbot snap itself is the real source of truth about what packages are already available and can be excluded from the plugin snap. Alternatively, I think we'd need something like a separate file to list the installed packages which plugin snaps could use to know which packages they can exclude.

[basak]

What cases do you have in mind where you think this would not work?

I think I had in mind cases where the required plugin dependencies collide/conflict with certbot's required dependencies. However then of course the plugin wouldn't work just from pip without snap, so perhaps that's not an issue at all.

Another example: if a third party plugin were to use augeas (and the main certbot snap did not), it'd have a dependency on libaugeas as well as the augeas lenses in /usr/lib/ somewhere. A plugin snap build wouldn't automatically know that those directories must not be stripped. This would have to be handed manually by the plugin snap developer, but it wouldn't make anything impossible I don't think.

To set this up, are you imagining the plugin including a subset of the directories under $SNAP/lib/python3.5/site-packages?

Right. At the same path, but with most of the other directories removed.

Is there any (easy) way to load the Certbot snap or its files into the build environment of the plugin snap?

There's build-snaps which might work. I think this might give you the installed snap available in the build environment only. At that point you could examine /snap/certbot/current. This might need some thought wrt. the version you're getting at build time versus the intended production version the snap will be intended to work with.

[bmw]

if a third party plugin were to use augeas (and the main certbot snap did not), it'd have a dependency on libaugeas as well as the augeas lenses in /usr/lib/ somewhere.

If the main Certbot snap did not contain augeas but the plugin snap did, do you know if the plugin would be able to dynamically load augeas? Would paths be set up properly for this to work while the code is being run by the Certbot snap? I haven't looked into the details of how snaps (and especially classic snaps) manage to keep their code isolated from the rest of the system.

There's build-snaps which might work. I think this might give you the installed snap available in the build environment only. At that point you could examine /snap/certbot/current. This might need some thought wrt. the version you're getting at build time versus the intended production version the snap will be intended to work with.

Thanks for the link! With some tinkering this seems promising to me. It seems like problem of different versions (and the fact the Certbot snap masks dependencies installed in the external plugin snap) could be largely solved by making it clear what dependencies will be installed in the Certbot snap and announcing any changes that we expect to break things such as ABI changes well in advance. Since most of Certbot's dependencies are entirely written in Python and their APIs have had few significant changes over the past 4+ years, I wouldn't expect there to be too many problems here.

[bmw]

I hackily moved this conversation to https://github.com/certbot/certbot/issues/7946. Please add any new discussion there.