[Specs] More advanced API specification format

[Mickael-van-der-Beek]

A few months ago I discovered that Node.js had a JSON version of it's API documentation. e.g:

http://nodejs.org/api/crypto.json

The format itself is decent but I think that it could go even further. I feel like if the params field would be reworked, using the JSON API docs and their different versions as data source for various modules would be really easy.

Some use cases I had in mind:

• building a "debug" module that validates parameter types / formats and outputs very verbose run-time errors if the specified input doesn't follow the spec.
• writing dynamic tests for the Node.js core itself that fuzzes the native methods and detects if errors are correctly thrown in case of invalid parameters
• custom error handling policies where the end-user (developer) can choose to handle all different types of errors through callbacks rather than exceptions (for invalid inputs).

https://github.com/joyent/node/issues/8249 https://github.com/joyent/node/issues/8390

I imagine that a whole range of other possibilities would be opened by implementing this feature and improving the specs.

The following format could be one way of expressing a parameter spec:

{
    "name": "filename",
    "optional": "false",
    "type": "String",
    "format": ["notEmpty", "isPath"]
}

Where notEmpty and isPath are validator functions that could possibly be added to the utils core module.

I don't think the change itself would be huge nor time consuming after the initial coverage of the current API's.

Since this doesn't involve writing C/C++ code, I'd be willing to work actively on this feature ! Obviously, this could be done by an exterior NPM module but in my opinion a feature like a detailed spec / spec validation should be centralized in the core repository.

What do you guys think ?

[bnoordhuis]

writing dynamic tests for the Node.js core itself that fuzzes the native methods and detects if errors are correctly thrown in case of invalid parameters

That's not a bad idea! Although the test suite should cover the API surface pretty well, more validation won't hurt.

I do see a practical issue though: there are a fair number of API methods that allow more inputs than what is documented for backwards compatibility reasons. I'm not sure how many exactly but I imagine you could end up with a fuzzer that's a big pile of special cases.

[Mickael-van-der-Beek]

• Parameter overloading could be solved by having a list of possible parameter combinations. Here's an example in MongoDB's native Node.js driver docs:

http://mongodb.github.io/node-mongodb-native/api-generated/collection.html#find

• Backwards compatibility could be partly handled by versioning the specs in relation to the Core and then flagging the special cases as being linked to a particular past version.

[darrenderidder]

@Mickael-van-der-Beek - I like it. Another potential use case that a lot of people on the node group really wanted was an alphabetical API reference like Angular, et al, rather than just the man-page style. Last time I checked, the json docs didn't have quite enough consistency to build an API reference automatically.

[Mickael-van-der-Beek]

@darrenderidder - Thanks! An alphabetical spec would definitely be a cool improvement!

Currently the JSON and HTML API specs are generated with a non-NPM-published module called node-doc-generator developed by @isaacs.

I feel that although it's very convenient, I'd prefer to do it the other way around. e.g: JSON API specs generate HTML and maybe also let description strings be markdown that is than converted to HTML.

Here is an example of the schema syntax I have in mind:

{
    name: 'fs',
    type: 'module',
    methods: [{
        name: 'readFileSync',
        type: 'method',
        signatures: [{
            name: 'filename',
            type: 'String',
            format: [
                'notEmpty',
                'isPath'
            ]
        }, {
            name: 'encoding',
            optional: true,
            default: 'utf8',
            type: 'String',
            format: [
                'notEmpty',
                'isEncoding'
            ],
            deprecated: '0.10.25'
        }]
    }]
}

The type key, when not used in a signature description, can take three possible values: class, module and method.

The deprecated key is an optional flag that displays a visual warning to the user reading through the specs to underline that it is not recommended to specify this parameter.

I have already written and tested (86% coverage) a spec validator module in a new repo I created called Node Cerberus: Mickael-van-der-Beek/node-cerberus@864f93ac2a853a94325289fb5994e19b8086f991.

It supports type and format validation like above but also various options that can be configured at the module level or spec level:

• typeStrict (default=true): validation should fail in case the type doesn't match the spec
• formatStrict (default=true): validation should fail in case the format doesn't match the spec
• existenceStrict (default=true): validation should fail in case a parameter is not specified (like a global { optional: true })
• nullAsExistence (default=false): null is considered as an existing (as in not missing) value
• undefinedAsExistence (default=false): undefined is considered as an existing (as in not missing) value

If you want more details about the spec format for nested Objects / Arrays or wildcard Array element validation, PM me or look at the tests for node-cerberus. I'm sorry, I didn't have the time to write documentation yet.

The fuzzer module, in the same repo is still a work in progress but I already created a directory containing a list of "special" values for the Node.js + JavaScript native type, that will be used as payloads during fuzzing tests. Link below:

Mickael-van-der-Beek/node-cerberus/src/fuzzer/payloads

What do you guys think ?

PS: One case I'm currently not handling in my validation spec format is linked parameters. e.g:

function MyMethod (fileData, fileEncoding)

where fileData is expected to be a String encoded using the fileEncoding encoding.

One possible solution for this:

signatures: [{
    name: 'fileData',
    type: 'String',
    format: [
        {
            name: 'isEncoding',
            args: ['ref:fileEncoding']
        },
        'notEmpty'
    ]
}, {
    name: 'fileEncoding',
    type: 'String'
}]

Which means that a format could be specified as a String (representing a validation method name) or an Object containing a name key for the method name and an Array of args that will be applied to the validator.

The 'ref:fileEncoding' argument here would be the fileEncoding variable value at validation time.

It's a bit of a hack so I'm definitely open other ideas !