Feature request/roadmap: Container/image to update and add a docker marketplace

[GuiPoM]

Hello,

First, thanks a lot for all the efforts that were made to make openmediavault a better place when you are using docker and docker compose. This is still an early phase for this feature, but it already covers lots of use cases and seems now mature. Just for sharing that info, I will publish a video tomorrow (in French) on how to migrate and what are the benefits on the long term.

Talking about long term, I have two feature requests that would make the plugin even better, and as I can't find a kind of roadmap here or on the forum (but maybe it exists, I did not find it !), here are my two requests:

• I am used to the feature on portainer ee (business edition) that display a light indicator next to an image, a container and a stack (compose) to indicate if there is any new image update released comported to the ones used by the compose file. This helps a lot to identify what shall be updated, compared to diun or watchtower that are of course able to notify, but not integrated. This prevents auto update but keep a visual indicator on what is not yet up to date
• Is there a market place planned to hide for beginner users the compose tiles ? like the "+" button with examples, but without the need to understand first the concepts of docker and compose. I would love to see a very simple UI showing services you can (almost one click) install on your NAS like jellyfin, adguard, homeassistant with a summary on where it is running. Then, if you are an advanced user, you can easily switch to the current UI that displays everything and seems very useful for daily maintenance.

I hope these two feedbacks can be useful, you may already have think a lot about it and I like the approach of providing first all required technical features, this is more for a long term view of this service and have it more accessible. Thanks ! Guillaume

[ryecoaaron]

display a light indicator next to an image, a container and a stack (compose) to indicate if there is any new image update released

I have looked into this and have not found a good way to determine if an image is available. If someone points me to some way to do this, I will gladly add it.

Is there a market place planned to hide for beginner users the compose tiles ? like the "+" button with examples, but without the need to understand first the concepts of docker and compose. I would love to see a very simple UI showing services you can (almost one click) install on your NAS like jellyfin, adguard, homeassistant with a summary on where it is running. Then, if you are an advanced user, you can easily switch to the current UI that displays everything and seems very useful for daily maintenance.

This is a tough one. The web framework for OMV doesn't really allow for a dynamic change between a regular and advanced interface. I was also hoping the "Add from Example" would serve this purpose. While the user still has to deal with a compose file, the example should require very few changes. There are many examples and I think they cover some of the most popular apps/images. I am also hoping the user learns a little about docker and compose. I think portainer obscures too much and the user doesn't really understand what they are using.

[GuiPoM]

display a light indicator next to an image, a container and a stack (compose) to indicate if there is any new image update released

I have looked into this and have not found a good way to determine if an image is available. If someone points me to some way to do this, I will gladly add it.

I would love to, but I am not sure portainer-ee is open source, I don't know where the source code is !

What I know is that I took part to issue 1304: https://github.com/portainer/portainer/issues/1304 And implementation was performed in PR1082 and deployed under image portainerci/portainer-ee:pr1082

Is there a market place planned to hide for beginner users the compose tiles ? like the "+" button with examples, but without the need to understand first the concepts of docker and compose. I would love to see a very simple UI showing services you can (almost one click) install on your NAS like jellyfin, adguard, homeassistant with a summary on where it is running. Then, if you are an advanced user, you can easily switch to the current UI that displays everything and seems very useful for daily maintenance.

This is a tough one. The web framework for OMV doesn't really allow for a dynamic change between a regular and advanced interface. I was also hoping the "Add from Example" would serve this purpose. While the user still has to deal with a compose file, the example should require very few changes. There are many examples and I think they cover some of the most popular apps/images. I am also hoping the user learns a little about docker and compose. I think portainer obscures too much and the user doesn't really understand what they are using.

I am not so sure portainer obscures anything, and regularly sharing with a community of non tech viewers, even sharing a step by step tutorial is already too complex to follow, they want to copy paste a config, and don't think about it. I know that this is not a good idea, moreover when you deal with critical services. But if you think about it, many are running omv without knowing how a Linux OS is working, and no one will request that knowledge. Regarding network, storage, services ... So expecting the same for service provided through docker is to my understanding very similar, a zero knowledge approach should be viable.

I am sure you know it but I was thinking of the famous "custom applications" that you can find in TrueNAS Scale. Which is even more complex, as it is based on kubernetes. https://www.truenas.com/docs/scale/scaletutorials/apps/docker/ or the one from Synology DSM or similar But I can understand if they are restrictions from what is possible from OMV UI. An Application Catalog or Market, replicating the list you proposed but located as a root entry, with an icon, an installation status, with very few mandatory steps to finalize the template, could be very a game changer for people having no knowledge.

As you already provided configuration to store data, backup config and user data, install/upgrade is almost the remaining piece to easily deploy with basic knowledge. And in case of any issue, you can still rely on the full UI and the community to solve you issue.

[ryecoaaron]

I will look again for ideas on the updated image available indicator.

regularly sharing with a community of non tech viewers, even sharing a step by step tutorial is already too complex to follow

We support some of those same users on the forum as well. Personally, my mind is not able to think on the no-knowledge level and I struggle with people who have no desire to learn. The plugin is about as simple as I can write with the framework and with my simplifying ability. Much simpler and it would make it a pain to use for advanced users.

I am sure you know it but I was thinking of the famous "custom applications" that you can find in TrueNAS Scale

Nope, never used it. But I'm sure they based it on kubernetes because Scale is meant to run on multiple nodes. That is something you can do when you have hundreds of people working on it.

An Application Catalog or Market, replicating the list you proposed but located as a root entry, with an icon, an installation status, with very few mandatory steps to finalize the template, could be very a game changer for people having no knowledge.

I'm not sure I can make the Add from Example any easier. You select the example and give it a name/description. Click add and then edit it for paths/users/etc. I can make a quick video on that if you would like.

[ryecoaaron]

I came up with a way to determine if a new image update is available but it only works for docker hub images. I haven't been able to find an unauthenticated way to get lscr.io images. The change still needs work. https://github.com/OpenMediaVault-Plugin-Developers/openmediavault-compose/commit/104303fae826419239053bf80dc428581ddcf522

[GuiPoM]

That's look promising. My only concern is about rate limit, I can't see if and where the request would be cached to avoid lookups any time you lists them. But maybe this is handled by the omv UI framework, I have no knowledge here.

[ryecoaaron]

It isn't cached. I would have to write all of that from scratch. I do not want to do that since nothing in the images tab is stored in a database or file. I would have to store the status in a file and try to update the file whenever anything in any tab updated the image. Too much to maintain for a feature that is not super important.

[GuiPoM]

I can imagine that it is more complex. And I know that you don't find the feature that important. So thanks for spending some time on it. Having shared how to deploy services like diun or watchtower, viewing that info and seeing it easily accessible close to the container and image is very useful even for power users. Beginners have also a clear view on what is outdated. There is also in the same spirit the unused aspect of an image. No container is using it. It helps a lot to safely clean the system. I can understand these are not top priority features, but if and when people switches from a graphical tooling like portainer to this plugin, having such help is more than welcome for dialy or Weekly maintenance!

[ryecoaaron]

I'm not done working on it. I've spent too much time on it to throw it away now. I just don't want to add caching.

For my own systems, I just click pull for my compose files. If it downloads a new image, I know I should restart the container at some point. If it told me there was an update, my process would be exactly that same.

[GuiPoM]

I prefer not changing my live services at all. Even if pulling old images is still an option, I first check changelog and impacts to verify if this is a maintenance (like lsio is doing a lot) or an update. The last nextcloud 27 update was a complete disaster, so I am now even more careful with these preliminary checks. But I guess your routine can easily be applied when you are comfortable with docker and command line. Again this is more for people used to UI and not very trained to docker commands.

[ryecoaaron]

Do you use the 'latest' tag on your live services?

My routine works perfect with the compose plugin. You select a file in the Files tab and then click Pull. You will see if it pulls a new image. If it does and you want to use it, you just click the Up button. No command line and you really don't have to understand docker to do this.

[ryecoaaron]

6.9.6 is in the repo. The status might need a little work still but it seems to be working for the most part.

[GuiPoM]

Hello,

It seems to be somehow working. The status is of course not yet accurate compared to Portainer:

I often use latest tag, but for some services I can switch to tagged versions to avoid any unwilling upgrade.

PS: I have published my video some hours ago: https://youtu.be/EZOvY5di1H0

[ryecoaaron]

my code only checks docker hub. I couldn’t find an api that doesn’t need auth for ghcr or lacr

[GuiPoM]

I have no idea in details but I am using the vscode docker plugin which has such capabilities and I know that some catalog registries have been added there: https://github.com/microsoft/vscode-docker/issues/869

For github this is basic auth if I remember correctly and needs user login and password. Maybe this could be added as an advanced config if and when you have some time to enhance this feature.

For lsio I thought there were available through github, so it should also solve that part.

I am using VSCode with remote SSH and docker extensions on remote host. It also lists images with status:

[alexshalex]

you can change lscr.io/linuxserver to linuxserver and do a pull from docker hub

[jumkey]

Hello @ryecoaaron

This part should be able to be replaced by the https://docs.docker.com/engine/api/v1.45/#tag/Distribution API.

https://github.com/OpenMediaVault-Plugin-Developers/openmediavault-compose/blob/4a9b87a6dfc5f7d3a02af1f50b58c7bead0048c6/usr/share/openmediavault/engined/rpc/compose.inc#L1688-L1725

curl --unix-socket /var/run/docker.sock http://{hostname}/{version}/distribution/{name}/json
# version>=v1.30

e.g.

curl --unix-socket /var/run/docker.sock http://localhost/v1.45/distribution/lscr.io/linuxserver/lychee:latest/json
curl --unix-socket /var/run/docker.sock http://localhost/distribution/redis/json

php https://gist.github.com/FergusInLondon/c3eb96f1f6565a81e509ece6b14b9a78

[ryecoaaron]

@jumkey thanks for the pointer. That could be very helpful. I will see what I can come up with.

[ryecoaaron]

Hitting the docker api didn't work without pulling the image. But in the process of testing changes, I found regclient/regctl and it works will all repos. 7.1.5 is in the repo and the image status should now work with all image registries.

[ryecoaaron]

Closing this issue because the image status should be working now and the examples are as close to a marketplace as the plugin will get.

[GuiPoM]

Thanks a lot for your efforts @ryecoaaron :)

I am close to being able to replace portainer ! that was one of my last pain points

[GuiPoM]

Closing this issue because the image status should be working now and the examples are as close to a marketplace as the plugin will get.

May I ask you to use green color for positive information (current) and yellow for warning (available) ? Thanks !

[ryecoaaron]

https://github.com/OpenMediaVault-Plugin-Developers/openmediavault-compose/commit/3ee6e12679393c293cb36760eb7dc0e45593cf4e

[ryecoaaron]

I had another change to make so I pushed 7.1.6 to the repo.

[GuiPoM]

3ee6e12

Perfect ! Again, thanks a lot for the time you spent on this topic !