gorenje/node-red-contrib-nodedev

Node-RED nodes for the development of Node-RED nodes in Node-RED using Node-RED nodes.

What?

The idea behind this collection of nodes to democratise the development of Node-RED nodes. Normally the development of your own nodes will require the use of a third-party editor. Something like VSCode or Vim or Atom or Emacs. This requirement makes creating nodes that much harder. So why not create nodes for Node-RED in Node-RED? After all, if you are using Node-RED, you probably understand how to use Node-RED.

So this package tries to provide some supporting nodes for making node development in Node-RED possible and simpler. I have created a set of nodes that fulfill my needs, everything else is bound my imagination.

Imagine a continous integration server combined with an editor. This can be done because Node-RED can both editor code (using something like a function node) and also perform actions such as posting data off to servers (http request node).

What do these nodes provide?

A presentation a nodes package collection in the form of nodes. The nodes that represent a file in the package are called PkgFile and are a template representation of the files content. Template as they parsed by mustache so that {{ replaceme }} will be replaced. But this can also be deactivated using the syntax specification.
A NpmTarball node will create a gzipped tarball for uploading to npmjs.com where all nodes live. This does not need to have since nodes can be installed localling into the Node-RED instance.
A NRInstall node allows for installing the node package into the Node-RED instance in which the flow is running. This means that nodes can be created, tested and modified with Node-RED. There are a few tricks so that Node-RED does not need to be restarted upon updating the package, more below.
A NodeFactory node that can create templates for node development. It also converts existing .tgz packages to PkgFile nodes, meaning that existing packages can easily be ported to this style of development.
A NpmPublish allows node packages to be published to a node registry, for example npmjs. The NpmPublish can also be used to publish to private registries, for example, those based on Verdaccio.
A OTPGenerate node can be used to generate an One Time Password (OTP) for publishing nodes to registries. For example the NPMjs.com registry.
The NodeDevOps node controls what is done, i.e., install package locally, commit to GitHub or publish to Npm. For these operations to work, a support flow needs to be installed in Node-RED. This can be done using the FlowHub nodes.

This is all very confusing?

Of course, all this is very meta and it gets worse since these nodes are maintained by this flow. These nodes are bootstrapped in Node-RED to aid Node-RED node development within Node-RED. On the other hand, Node-RED node development is initially also confusing (requiring at least three different files). That's why the NodeFactory provides templates for various types of nodes.

This is not an inbuilt extension of Node-RED and obviously a more integrated workflow would be simpler. This is a attempt to find a better solution to Node-RED node development (not NodeJS development), one by which testing and fixing nodes becomes faster because everything is done in Node-RED.

Screencast

For a detailed explanation on how to use these nodes, check out the screencast presenting the NodeDev workflow.

Motivation

A comparison of Node-RED extensions and indirections

	Link Call	Subflow	Link Nodes[^4]	Custom Node	Function node	Flow Tab
Outputs [^1]	=1	>=1	=1	>=1	>=1	-
Embeddable[^2]	Yes	Yes	-	Yes	Yes	Yes
Modifiable[^3]	Yes	Yes	Yes	-	Yes	Yes
Packages[^5]	-	-	-	Yes	Yes	-
Reusable[^6]	-	Yes	-	Yes	-	Yes
Button[^7]	-	-	-	Yes	-	-
Iconification[^8]	Yes	Yes	Yes	Yes	Yes	-

What this package does is make custom nodes modifiable within Node-RED.

[^1]: Number of outputs definable. Subflows can have many outputs, link-call nodes only ever have one output. Zero outputs is possible for all extensions, one is standard and more than one is difficult. [^2]: Embeddable means with the functionality can be included in a flow. Link-in/-out nodes redirect the flow of messages without those messages returning to the point where they left the original flow. [^3]: Does Node-RED provide support of modifying the extension? Node-RED does not provide any support for modifying custom nodes within Node-RED. [^4]: A combination of Link-in and Link-out nodes. [^5]: Can external NPMjs packages be included in the extension? [^6]: Is there a structured form of reusability for other flows? Subflows are structured to be reused, function nodes are not. [^7]: Can the extension be triggered with a button, i.e., as an inject node? [^8]: Can the icon be customised to make for simpler identification?

Examples

Examples are included:

Converting an existing tar-gzip package file into a collection of PkgFile nodes for testing and maintaince and development --> flow
NodeFactory example of creating a boilerplate node for inclusion in a node package that can be installed into Node-RED --> flow
The FlowHub node package is developed using the NodeDev nodes
Also the introspection node package uses the NodeDev nodes

Config node with sidebar

The following animation shows how the example of a config node with sidebar button works:

This animation shows the development and installation of a palette node:

Importing existing packages

Using the Package Import sidebar, it is also possible to install existing packages to find out how others have built their nodes.

The gif shows some of the features:

import existing package,
view the source files in Node-RED,
preview of file content is shown in the info box,
installing package locally is possible using the NodeDevOps node.

This animation shows those features:

Of course making changes to the PkgFile nodes and then reinstalling the package will reflect those changes. This makes it simple to take an existing package, make modifications to suit oneself and then push those changes upstream to the original developer(s).

Tips

How to avoid restarting Node-RED?

Normally when developing nodes for Node-RED, a restart is necessary to flush older versions of the nodes out of Node-RED. This can be avoided by renaming the package each time its installed into Node-RED. There is no need to rename nodes, just the entire once in the package.json. Something I do is use a random number as extension to the package name. The workflow then becomes:

Create nodes
Install nodes
Using the palette manager to remove the node package
Reload the browser since it too has a cache of node code
Make modifications to the nodes
Change the package name
Reinstall the nodes
Repeat from step 3

Since all this happens in the browser, there is no leaving Node-RED, making it more efficient to test and modify nodes. Renaming of packages can be done automagically as demonstrated by the flow that maintains these nodes.