Upgrade to Node 18

[Pezmc]

Description

FlowForge currently uses Node 16 across the board.

Node 16 leaves security support in three months (11th Sept 2023). Node 18 is in long term support (LTS) until April 2025.

FlowForge should be upgraded to node 18 across the board before September. And again to Node 20, once that enters LTS at the end of 2023.

There are at least the following parts to this:

• [ ] Upgrading docker images
• [ ] Update Kubernetes helm charts
• [ ] Upgrading development environment
• [ ] Updating docs
• [ ] Update engines flags to >18 for all repos

Known repos so far

• [x] docker-compose/flowforge-docker
• [ ] flowforge/.github
• [ ] flowforge/flowforge-device-agent + .github
• [ ] flowforge/flowforge-driver-localfs + .github
• [ ] flowforge/flowforge-file-server + .github
• [ ] flowforge/flowforge-nr-file-nodes + .github
• [ ] flowforge/flowforge-nr-launcher + .github
• [ ] flowforge/flowforge-nr-persistent-context + .github
• [ ] flowforge/flowforge-nr-project-nodes + .github
• [ ] flowforge/flowforge-nr-tools-plugin + .github
• [ ] flowforge/flowforge + .github
• [ ] flowforge/forge-ui-components + .github
• [ ] flowforge/installer
• [ ] flowforge/usage-ping-collector
• [ ] Helm Tests coverage
• [ ] Our own server infrastructure:
  • [x] Kubernetes/file-server
  • [x] Kubernetes/flowforge-container
  • [ ] Kubernetes/flowforge-device-agent

Epic/Story

No response

Have you provided an initial effort estimate for this issue?

I have provided an initial effort estimate

[Pezmc]

@hardillb I'll need some help from you identifying all the places (i.e. our servers) that the practical upgrade needs to happen to add to the list above.

[hardillb]

We should already be tracking this in https://github.com/flowforge/flowforge/issues/2181 iirc

[hardillb]

All running prod/staging system run containers built from the helm project.

Updating the Dockerfiles for those will automatically roll into Staging/Production. With note that instances running on existing stacks will not change.

[MarianRaphael]

Copy of @knolleary description from https://github.com/flowforge/flowforge/issues/2181 (closed)

We currently base our stacks on node.js 16. We need a plan to support node 18 - particularly as we have seen cases of nodes that only support node 18 or later (eg node-red-contrib-salesforce).

We cannot just update all exists to Node 18 as that could break existing instances if they have nodes that don't support 18. I think this is rare, but not impossible. Should the choice of node.js version be something the user is given? with our existing stack model, we could create two sets of stacks for each Node-RED version - one node 16 and one node 18. However for many users, the choice of node.js version is irrelevant and this would force them to pick from a long list. we could role out node 18 with just the Node-RED 3.1 stack. Users on existing stacks will stay on Node 16 and they can make a more considered decision to update their stack to Node-RED 3.1/Node 18. If something doesn't work, then can change they stack back to the Node-RED 3.0.x/Node 16 one. We need to test/validate node 18 in the stack. All of our nr-launcher and plugin code will run in that version of node - we currently only test on node 16. So whilst the stack work described above is mostly admin tasks in the platform, we need to do some engineering work to validate our code on node 18.

The one breaking change between node 16 and node 18 that we've seen come up in some unexpected contexts is the dns resolution order has changed. Previously it would return ipv4 addresses first. It now returns them in whatever order the dns resolve provides them in - often resulting in getting an ipv6 address pack. That can cause unexpected issues. We have done some future proofing against that in the launcher by using 127.0.0.1 rather than localhost - but we need to fully validate that.

[hardillb]

The one breaking change between node 16 and node 18 that we've seen come up in some unexpected contexts is the dns resolution order has changed. Previously it would return ipv4 addresses first. It now returns them in whatever order the dns resolve provides them in - often resulting in getting an ipv6 address pack. That can cause unexpected issues. We have done some future proofing against that in the launcher by using 127.0.0.1 rather than localhost - but we need to fully validate that.

Both Node-RED, the nr-launcher have the following code at the start:

const NODE_MAJOR_VERSION = process.versions.node.split('.')[0]
if (NODE_MAJOR_VERSION > 14) {
    const dns = require('node:dns')
    dns.setDefaultResultOrder('ipv4first')
}

This forces the NodeJS 16 DNS order behaviour.

We should decide if we should do this for the device-agent as well