Closed knolleary closed 3 years ago
I have spent some time looking at the Flow Manipulation API and how the current rules work.
I have also spent some time looking more at how eslint rules work.
Lots of our rules want to check nodes of a particular type - such as the Function node, or HTTP In node. At the moment, every rule has to iterate over the complete Node list to find the nodes it cares about. I think we can do better.
With eslint, a rule can register a function for certain events during the parsing. For example, it can register a function to be called for every if...
statement.
I suggest we implement a similar approach.
The check
property is passed the config and context objects. It returns an object that provides functions that should be called by the linter based on the key value in the object.
type:<node-type>
(eg type:function
) it will get called for each node of that typeflow
it will get called for each flowsubflow
it will get called for each subflowgroup
is will get called for each groupnode
it will get called for each flow nodeconfig
it will get called for each config nodestart
it will get called, with the full flow object, before any of the above functions are calledend
it will get called, with the full flow object, after any of the above functions are callednamed-function
module.exports = {
meta: {
type: "suggestion",
severity: "warn",
docs: {
description: "ensure all Function nodes have a name"
},
schema: []
},
check: function(context, ruleConfig) {
return {
"type:function": function(node) {
if (!!!node.config.name) {
context.report({
location: [node.id],
message: "Function node has no name"
})
}
}
}
}
};
unique-http-in-paths
This example checks that all HTTP In nodes have unique paths. It uses the start
function to initialise its own state, the type:http in
to visit each node and capture its url and the end
function to check for any with more than one node using that url.
module.exports = {
meta: { ... },
check: function(context, ruleConfig) {
let seenPaths;
return {
"start": function(flowConfig) {
seenPaths = {};
}
"type:http in": function(node) {
if (node.config.url) {
seenPaths[node.config.url] = seenPaths[node.config.url] || [];
seenPaths[node.config.url].push(node.id);
}
}
"end": function(flowConfig) {
for (var url in seenPaths) {
if (seenPaths[url].length > 1) {
seenPaths[url].forEach(id => {
context.report({
location: [id],
message: "Duplicate HTTP In url"
})
})
}
}
}
}
}
};
The key benefit here is the linter can do one iteration over all of the objects and call the lint rules as needed - this will be much more efficient than each rule searching for itself.
For the rules that need to see the 'big' picture, they can use the start
or end
event with full access to the flow configuration.
I understand I am presenting a lot of changes here - but I hope you can see the value they will bring.
I will raise a new PR tomorrow (that builds on top of this PR) that implements this suggestion so you can see it in action.
Thank you for creating the PR. I think this is a great improvement. I'm taking a look at it little by little...
Regarding "5. package.json changes": nrlint-plugin-func-style-eslint
uses eslint4b
package, and the package should installed before webpack is called. Should we add some installation process on build-func-style-eslint-rules
script?
I think we should consider splitting this one repository into three and move away from having multiple modules in here.
nrlint
-> contains the core of the linter including all the 'core' rules as a single module.nrlint-plugin-func-style-eslint
-> because of its larger size@node-red/flow-parser
Having said that, I am wondering if splitting func-eslint out is a good idea or not. Once bundled, it isn't as big as I thought it would be. It might be cleaner to keep all of our rules the nrlint
bundle.
But regardless of that, to answer your question, yes will want to have build
script that takes the source and creates the files that get npm published. End users should not need webpack installed if they are just installing and using the linter from npm. Webpack should only be a devDependency for people who want to help develop the rule.
This is quite a large refactor that covers a number of items to be considered. I'm sorry about the size of this.
1. Moved Flow Manipulation API to its own npm module
I have moved the api to
@node-red/flow-parser
. I propose to eventually move it to its own repository. This allows us to require the API in both the CLI and WebWorker version of the API, rather than have to maintain two copies of it.I have some separate feedback on some of the behaviour of the API - for example, the way it handles subflows and that it doesn't know about
groups
.2. Restructure rules
I am proposing a slightly different file structure for modules that provide rules, as well as a way for rules to share metadata such as available configuration options. This is very influenced by eslint rules.
For example,
nrlint-plugin-core
now looks like this:each rule is in its own file under
lib/rules
.lib/index.js
is the main entry point to the module. It re-exports all of the rules:Each rule module should export an object as follows:
meta.type
- what sort of rule is this. eslint hassuggestion
problem
andlayout
meta.severity
- the default severity this rule reports at (can be overriden in config)meta.docs.description
- one line description of the rulemeta.schema
- a JSON Schema of the valid configuration options for this rule. Will can use this to auto generate a settings panel in the editor (future feature).check
- the function that will get called to run the ruleThe
check
function behaviour is slightly different.flow
,config
,context
.context.report
- this is the same model used by eslint.location
- an array of ids of the affected object (they wereids
in the previous version)message
- a message to display. (future: as with eslint, this could be a message-identifier to points to a message in themeta
object, allowing for i18n).severity
- optional property ofwarn
orerror
to override the configThe
lib/plugin.html
file is the plugin file loaded by the editor. It contains:worker
property that names the path to the worker file for this plugin.Each plugin has a webpack.config.js
lib/index.js
toresources/nrlint-plugin-core-worker.js
. The name of the output is not important as long as it matches theworker
property inplugin.html
library
configuration means all of the rules exported bylib/index,js
get added to the global variablenrlint.rules
.webpack
should be adevDepdendency
- part of the npm release process will involve running the build and publishing the version with the pre-build resources files in.3.
nrlint-main
nrlint-main/lib/linter.js
is the heart of the linter. It does the work to run the rules using the provided configuration and flow. This is a module that is used in both the Worker and the CLI version.nrlint-main/plugin/nrlint.html
is the main editor plugin that gets loaded. It starts the Worker and sets up theonmessage
handler to update the UI when the linter provides results back.nrlint-main/lib/nrlint-worker.js
is the Worker source code. It handles the communication with the editor.I have changed the main
linter.lint
function to not be async - as we don't run it as async in the editor anyway. If we need the ability to have async rules, we should think about how to introduce that back in cleanly.4. Configuration format changes
I'm proposing a slightly different format to the nrlintrc.js file.
plugins
section at the top lists the plugins that should be loaded. This is only needed by the CLI version.rules
property is an object that mapsrule name
to its configuration.true
enables that rule if there are no other config options needed"off"
will disable it (because a rule may want to treatfalse
as a valid setting that means the rule still runs but checks for something else)severity
property to override the rule's default severityI think this flatter structure will be easier for the user to not have to worry about which rule is provided by which module.
5. package.json changes
I've removed the
prepare
tasks as I think the webpack stage is a build-time action before being published to npm.I've added scripts to build each of the modules and copy their output to the top level
/resources
directory. This is helpful at development time, but ultimately I think that will have to change so younpm install
each sub module individually, rather thannpm install
the top level module into NR.6. TODO: None of the tests work
As I write this now, I realise I haven't updated any of the tests to work with these changes.
Next Steps
If there are any of these changes you disagree with, then please do say. I may have got slightly carried away here and this turned into a bigger refactor than I intended.
If you are happy with these proposals, the next steps will be:
@node-red/flow-parser
to its own repositorynext()
go into the subflow, or step over the subflow.