search

thin-edge / thin-edge.io

The open edge framework for lightweight IoT devices
https://thin-edge.io
Apache License 2.0
219 stars 54 forks source link

Support for installing packages from custom repository #1506

Open mjj29 opened 1 year ago

mjj29 commented 1 year ago

Is your feature request related to a problem? Please describe. I would like to be able to use the software deployment repository to deploy packages from a custom apt repository. I have multiple interdependent packages with dependencies into the main repository, so I can't just upload a .deb file. apt install will work, but only if you've configured my custom repo first. Multiple software repository entries we're not convinced will necessarily be processed in the correct order to install them.

Describe the solution you'd like A single Software Repository entry which deploys the named package and all its dependencies from both the custom repo and the standard repository, in some fashion.

Describe alternatives you've considered Options I've thought of so far:

If the URLs (in the .sources file) are HTTPS then having the apt plugin mark them as trusted, so they don't have to validate the GPG key is a little more reasonable, since they should be validated against the SSL PKI instead

Additional context

reubenmiller commented 1 year ago

Thanks @mjj29 for the suggestion. We are indeed investigating different forms of packaging where one is include hosting the debian (.deb) files in a repository making the installable via apt-get install.

We are still evaluating the options as we would like to get it right the first time and not have to change again in a few months time.

reubenmiller commented 1 year ago

Duplicate #1560

mjj29 commented 1 year ago

Hey, which ticket is this a duplicate of? I'd like to be able to follow the development of this feature

reubenmiller commented 1 year ago

It is #1560 ~#1506~ (I mentioned it when closing it). Idea is to get a better idea on what work is still required and having multiple tickets makes it a bit more confusing to track…hope that is ok

mjj29 commented 1 year ago

This is 1506?

reubenmiller commented 1 year ago

Apologies, my fault. The correct ticket is #1560

mjj29 commented 1 year ago

It's not quite the same as 1560 - that talks about the official thin-edge.io assets being in an apt repository - which could reasonably be implemented by manually configuring the repository before installing. However, this is to change the apt deployment plugin so that additional software can be installed from an apt repository that may be different to the manually configured one, through the cumulocity UI.

reubenmiller commented 1 year ago

Ah yes of course. For some reason I totally misinterpreted your ticket.

Though I’m fairly confident that this could be supported by use of the configuration plugin to manage the apt source lists under /etc/apt/sources.list.d/. As the apt plugin should pull in the required dependencies as long as the apt sources are configured before hand.

reubenmiller commented 1 year ago

And you can always create a meta Debian package which just references the packages you would like to install if you want that convenience factor

mjj29 commented 1 year ago

This is what I did for the short term, but it's still two steps and still relies on the c8y software management always trying to install them in the right order.

One other related thing I've noticed is that right now you have to specify the full version of the package you would like, rather than 'whatever the latest fix is please' - which would seem better?

reubenmiller commented 1 year ago

Though there is the option to create a meta debian package (and there are easy to use tools out there like fpm which can be used to create debian packages). The meta package can then define the intended install order by using Pre-Depends fields etc.

One other related thing I've noticed is that right now you have to specify the full version of the package you would like, rather than 'whatever the latest fix is please' - which would seem better?

Yes I think a special version tag like latest would also be nice here. Technically if you send the operation using Cumulocity API you can send a blank version and the apt plugin will pick the latest available version, however the UI does not allow to specify a blank version so it is not very user friendly.

reubenmiller commented 1 year ago

@mjj29 I just create an issue to track the latest improvement idea. Could you please review it? https://github.com/thin-edge/thin-edge.io/issues/1594

mjj29 commented 1 year ago

BTW, I don't think that pre-depends works when the first package defines the repository that contains the second package. The issue here is the order that c8y's software management tool will give the installation requests to the device, not the installation order of the packages

reubenmiller commented 1 year ago

Yes true I don't think the Pre-Depends will work in this case, as if the package is providing the apt repository config, then an apt-get update needs to be run before you can install a package from it (and this is impossible to run in the maintainer scripts due to the existing apt lock due to the current apt installation).

Normally this use-case would be solved by something like Cumulocity's device profile handling. This would provide the possibility to combine configuration and software packages into one operation where the operation where you could define that configuration files should be installed before software packages.

reubenmiller commented 1 year ago

But current setup it would require you to apply the apt repository configuration first via apt or a configuration, then installing the package. Is this ok for now?