Nodered Projects Feature + flow_refresh sidecar

beasteers commented 10 months ago

Is your feature request related to a problem? Please describe.

NodeRed has a really nice feature for version controlling your flows called Projects (which lets you commit flow changes via the UI).

I'm considering managing nodered through a Dev/Production GitOps-style pipeline where:

the dev version has the Projects feature enabled and you make changes and commit to a git repo.
And then the production version (treated as a read-only nodered, could be headless) uses the sidecar to update changes from configmaps created from the git repo (possibly using git-sync).

I'm curious to know how the flow_refresh.py sidecar would interplay with the files generated by the Projects feature.

I ask because I noticed extra-node-modules.json which seems different to how nodered projects seems to manage things (through package.json) and I was wondering if you've considered these two features together.

Describe the solution you'd like

someone can make changes to a flow, add modules, etc. using the projects feature of the dev nodered instance and commit the changes via the UI.
I can have an automated workflow to pull the changes into the production nodered instance and automatically deploy flows and install node modules, etc so that the new version can be deployed automatically (without any interaction with the UI)

Describe alternatives you've considered

It looks like the flow_refresh.py file pretty much does this, but I am just curious about any potential pitfalls around combining these two tools.

Search

[X] I did search for other open and closed issues before opening this.

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Additional context

No response

dirien commented 9 months ago

I like this very much @beasteers!

hinrichd commented 6 months ago

@beasteers I think combining UI (1) and flow_refresh.py (2) would not work together. I use an UI/Dev Instance with a PVC to develop the flows and settings etc. Like an native instance on a single server. Your option 1.

Later I use the settings, flows etc. and pass them to the helm values together with the side car to install additional node modules from scratch. No matter on which node the pod is starting and with out any PVC. Or on an different cluster. Changes are only deployed with helm upgrade. The node-red interface should not be used (RO, Adminonly) otherwise the newly deployed settings from UI will be lost if the pod is starting on a different node. This approach should cover the kubernetes micro service architecture, with may node-red pods all deployed from main code basis.

Of course, the flow_refresh.py is no perfect at all. External npm packages, themes etc. can not installed this way, as the node-red api do not have this option. As you mentioned bevor an initContainer could be install npm packages like themes etc. before the container is starting. Actually I have not installed any npm packages via an initContainer.

hinrichd commented 4 months ago

U should not import flows from an config file with the sidecar when you have enabled the projects mode in node red. You will have two diffrent sources of truth.

If you clone or push from a remote, node red will not auto install node modules from the package.json aka Dependencies Tab but will alert the user. The user can install the node-modules from the dependencies tab by klick on install button.

The project mode needs an seperate sidecar, which is watching the packages.json an runs the flow-refresh.py on any changes. It should than parse the node modules from the package,json directly instead of the env extranodemodules. May be this could work. Or a node-red feature itself to autoinstall debendencies when pulling or cloning in projects mode.

SchwarzIT / node-red-chart