Docker images on the Hub are 9 months out of date / not regularly published even when there isn't a Node Red update

webash commented 1 year ago

I was asked to published this by @knolleary in this thread on the forums. Happy to try help with this issue if someone has the time to handover existing process to me.

What are the steps to reproduce?

Visit the Docker Hub for Node Red

What happens?

The latest published images are 9 months old.

What do you expect to happen?

The images should be republished as often as any downstream image (eg, Alpine) are pushed to ensure that updates in that dependency are included in the node-red images.

Please tell us about your environment:

[ ] Platform/OS: Docker

[ ] Browser: N/A

If you are a Portainer user, please attach a screenshot of your container details.

Containers -> Click on the node-red container -> Scroll to Container Details and take a screenshot

If you are a command line user please execute the following command and attach the log file.
```
docker inspect <container_id> > container.log
```

deschmih commented 1 year ago

I fully agree to @webash , node-red image should republished in a timely constant manner for solving critical and high CVE's . The actual image on docker hub contains many of high and critical CVE's which are allreday fixed by other's in past.

hardillb commented 1 year ago

OK, a few thoughts

Which versions do we respin? I'm going to say always only the last release, we are not going to respin back level versions
How often?
Do we need a numbering/naming scheme so we know what build people are on? Or do we just use the hashes and use the build date attribute?

deschmih commented 1 year ago

For me it would be fine to have the latest release without CVE's --> so respinning latest version would a good and handy soultion. Form my point of view a respin-period of 3-4 month would be resonable to react on security issues. Please fell free to add comments or other opinions.

webash commented 1 year ago

I think (as an outsider, but a long-term Node Red user) latest release of Node Red being the only regularly rebuilt version only is a fair position; people shouldn't expect support for older versions in an OSS project, and ultimately if they need an older version for some reason, they likely have a complex environment that should justify them building their own container on a regular basis (for upstream fixes), targeting whichever explicit older version (of node red) that they like. Average users will be the most vulnerable, and likely use :latest without knowing much about how containers work, nor about the security issues inherent with them. Thus, ensuring at least that :latest (and :current.version.number if it is separate image) is regularly rebuilt should cater for improving most scenarios.

Most actively developed open source projects with official containers seem to rebuild their containers (at least) weekly; which I imagine is fine if that is automated via Github Actions or similar; but if you're needing to do it manually due to complexities then I suspect that becomes unwieldly. Happy to help with automated rebuilds/pushes if that's the part that holds back more frequent rebuilds.

number/naming seems like a complicated space, so I'd forgive leaving that to a later time. Potentially my only preference would be to have :latest point to the :current.version.number image, so that the hashes match, giving those who care the ability to inspect such; but I'm coming at this suggestion with 'reckons' rather than experience.

hardillb commented 1 year ago

I have just re-spun the 3.0.2 docker container, it is using the same tags as when it was released (and is tagged as latest).

At this point it's going to be a manual process. The GitHub action that builds the containers needs a pretty major overhaul soon (due to using deprecated methods for setting variables for later steps). Once 3.1.0 releases I'll look at doing that and then see if we can safely automate the re-spin action.

Please give the new images a quick test

A-Ghorab commented 1 year ago

Hello @hardillb

This new docker version broke our build system as we were depending on manifest info provided by docker hub

for example node red 2 still have this manifiest info

https://registry.hub.docker.com/v2/nodered/node-red/manifests/2.2.3-12

this trimmed version just for info

{
   "schemaVersion": 1,
   "name": "nodered/node-red",
   "tag": "2.2.3-12",
   "architecture": "amd64",
   "fsLayers": [
   ]
}

but after the new release we only get OCI files without any manifest for details like node-red version 2 does

{
    "errors": [
        {
            "code": "MANIFEST_UNKNOWN",
            "message": "OCI index found, but accept header does not support OCI indexes"
        }
    ]
}

hardillb commented 1 year ago

@A-Ghorab None of the 2.x images should have been touched at all. Only the 3.0.2 containers where updated.

A-Ghorab commented 1 year ago

@A-Ghorab None of the 2.x images should have been touched at all. Only the 3.0.2 containers where updated.

yes i just took the version 2 as an example on how the previous output was like.

Node-red docker files for version 3 were the same but now they are showing the error i mentioned in the previous comment.

this happened when the new node red docker images were released.

hardillb commented 1 year ago

You are going to need to be a LOT more explicit about what isn't working here. Give clear examples of what is broken, not of things that work.

A-Ghorab commented 1 year ago

You are going to need to be a LOT more explicit about what isn't working here. Give clear examples of what is broken, not of things that work.

yeah my bad for not explaining it well. i suspect it might be an issue related to docker and not related to NodeRed entirely

ok i was using docker hub to fetch the node-red docker image details like the NODE_RED_VERSION

this was working fine until the images has been updated it doesn't show the details right now

steps on how i used to get them

get token from https://auth.docker.io/token?service=registry.docker.io&scope=repository:nodered/node-red:pull
call this url with token https://registry.hub.docker.com/v2/nodered/node-red/manifests/latest
read the result as json and fetch the needed info.

right now when i go through the steps i get an error

{
    "errors": [
        {
            "code": "MANIFEST_UNKNOWN",
            "message": "OCI index found, but accept header does not support OCI indexes"
        }
    ]
}

the docker image it self is working fine for me nothing to it but the manifest section is the one affected by the changes.

Thanks

hardillb commented 1 year ago

@A-Ghorab I don't think this is directly related to the images, it looks like it's a change in the Docker registry.

The error appears pretty explicit, you need to include the right Accept header in the request

e.g.

!/bin/sh
TOKEN=`curl 'https://auth.docker.io/token?service=registry.docker.io&scope=repository:nodered/node-red:pull' | jq -r .token`

echo $TOKEN

curl -H "Authorization: Bearer $TOKEN" -H "Accept: application/vnd.oci.image.index.v1+json" https://registry.hub.docker.com/v2/nodered/node-red/manifests/latest

A-Ghorab commented 1 year ago

@A-Ghorab I don't think this is directly related to the images, it looks like it's a change in the Docker registry.

The error appears pretty explicit, you need to include the right Accept header in the request

e.g.
!/bin/sh
TOKEN=`curl 'https://auth.docker.io/token?service=registry.docker.io&scope=repository:nodered/node-red:pull' | jq -r .token`

echo $TOKEN

curl -H "Authorization: Bearer $TOKEN" -H "Accept: application/vnd.oci.image.index.v1+json" https://registry.hub.docker.com/v2/nodered/node-red/manifests/latest

Yes that exactly what i did to get the needed result and it's returning a json object that is different than the previous one. (OCI json format instead of v1 manifiest)

and it doesn't include the history section that i used to relay on to fetch the node-red version from the image.

ok just to be clear on the issue i'm facing maybe you can guide me,

Is there a way to fetch the current node-red version and nodejs version used inside the docker image? as this is what I'm trying to get.

Thanks

hardillb commented 1 year ago

@A-Ghorab No idea, you'll have to explore what the Docker registry API offers

At this point this is not something the Node-RED team have any influence over, the imagers were built and pushed using pretty much the exactly the Docker build/push GH Action.

A-Ghorab commented 1 year ago

@hardillb yeah thanks for the help i did find a way out using docker hub :)

just one thing i noticed when i was checking the env for the images

[
"NODE_VERSION=16.20.0",
"YARN_VERSION=1.22.19",
"NODE_RED_VERSION=master",
]

the NODE_RED_VERSION is set to master shouldn't it be 3.0.2 ?

Thanks

hardillb commented 1 year ago

@A-Ghorab #376 should fix that when it gets merged I'll respin the builds again

hardillb commented 1 year ago

Containers rebuilt again, please check

BloodBlight commented 4 months ago

Just curious about the state of this. I was trying to get nodejs updated not realizing 16 was baked into the image. V16 is no longer the recommended version and there is at least 1 pallet that will not install due to the age of nodejs.

hardillb commented 4 months ago

There is a build available with nodejs 18 latest-18

The versions on nodejs will be updated with the release of - Node-RED v4. We can not change the version that is tagged as latest without a major version bump as it is a breaking change (node modules with native components will need to be rebuilt)

hardillb commented 4 months ago

I'm going to close this now as there is a mechanism to rebuild the latest releases to pick-up upstream fixes (e.g. from alpine) when required.

webash commented 4 months ago

In case I wished to selfishly 'borrow' such a mechanism myself for my own projects, could you share the commit / file changes made to get this working?

hardillb commented 4 months ago

@webash https://github.com/node-red/node-red-docker/commit/d6276872ad3ffd0a260b817e06dc4aebb56c1879

node-red / node-red-docker