[RFC]: Update `@stdlib/assert/*` benchmarks to measure affirmative/negative test values

[kgryte]

Many of the benchmarks in @stdlib/assert/* measure performance of an "average" test case, meaning that test values are a mixture of values resulting in true and false values.

An example package having such benchmarks is @stdlib/assert/is-string.

Preferably, we'd split benchmarks in separate positive/negative benchmark runs to have a better handle on performance depending on the input value type. While, for many assert packages, the difference is not likely to be significant, for others, the performance difference will be significant, as the assertion logic is more complex or may, in the attempt of robustness, perform various operations which are slow, such as walking the prototype chain or resolving a constructor name, etc. In these instances, having a more granular understanding of performance is useful, as downstream consumers can then make accommodations if such perf cliffs are undesired (e.g., when implementing dispatch logic and needing to resolve an input value's "type" before delegating to a specialized implementation).

An example package having benchmarks which are split into positive/negative test values is @stdlib/assert/is-complex-like.

The request in this issue is to refactor any assertion packages which are not split into positive/negative test values to be similar to @stdlib/assert/is-complex-like.

[performant23]

So, basically if I understand correctly, the benchmark should consist of separate positive and negative test cases for both primitive values and objects.

1. Primitives: Positive - Should return true when passed to the 'isString' function. Strings: ['5', 'beep']

2. Primitives: Negative - Should return false when passed to the 'isString' function. Non-strings: [5, NaN, true, false, null, undefined]

3. Objects: Positive - Should return true when passed to the 'isString' function. String objects: [new String('beep')]

4. Objects: Negative - Should return false when passed to the 'isString' function. Negative object values: Arrays: [[]] Objects: [{}] Functions: [function noop() {}]

A high level rundown of such a program considering the example of isString would be like above right?

[kgryte]

@performant23 Yes, your understanding seems correct.

[kgryte]

If you are interested in working on this, I suggest picking one package whose benchmarks should be updated and only focusing on that package.

[performant23]

Yeah, thanks for the advice! I'll pick up is-string right away since we know it has to be refactored!

[kgryte]

Great! Thanks!

[Snehil-Shah]

Hey @kgryte, can I pick a package and work on it too, if it's not specifically assigned to someone?

[kgryte]

@Snehil-Shah Sure. Maybe choose @stdlib/assert/is-boolean?

[performant23]

Hey @kgryte , I have refactored the benchmark for isString as shown below. The benchmark results are also attached herewith. I'd be really grateful if you could guide me with any feedback on this so that I can potentially move to other modules. To start with, I can take up all the modules starting with 'has' prefix (about 37).

isStringBenchmarks.txt

'use strict';

// MODULES //

var bench = require( '@stdlib/bench' );
var isBoolean = require( '@stdlib/assert/is-boolean' ).isPrimitive;
var pkg = require( './../package.json' ).name;
var isString = require( './../lib' );

// MAIN //

bench( pkg+'::primitives:positive', function benchmark( b ) {
    var values;
    var bool;
    var i;

    values = [
        '5',
        'beep'
    ];

    b.tic();
    for ( i = 0; i < b.iterations; i++ ) {
        bool = isString( values[ i % values.length ] );
        if ( typeof bool !== 'boolean' ) {
            b.fail( 'should return a boolean' );
        }
    }
    b.toc();
    if ( !isBoolean( bool ) ) {
        b.fail( 'should return a boolean' );
    }
    b.pass( 'benchmark finished' );
    b.end();
});

bench( pkg+'::primitives:negative', function benchmark( b ) {
    var values;
    var bool;
    var i;

    values = [
        5,
        NaN,
        true,
        false,
        null,
        undefined
    ];

    b.tic();
    for ( i = 0; i < b.iterations; i++ ) {
        bool = isString( values[ i % values.length ] );
        if ( typeof bool !== 'boolean' ) {
            b.fail( 'should return a boolean' );
        }
    }
    b.toc();
    if ( !isBoolean( bool ) ) {
        b.fail( 'should return a boolean' );
    }
    b.pass( 'benchmark finished' );
    b.end();
});

bench( pkg+'::objects:positive', function benchmark( b ) {
    var values;
    var bool;
    var i;

    values = [
        new String( 'beep' )
    ];

    b.tic();
    for ( i = 0; i < b.iterations; i++ ) {
        bool = isString( values[ i % values.length ] );
        if ( typeof bool !== 'boolean' ) {
            b.fail( 'should return a boolean' );
        }
    }
    b.toc();
    if ( !isBoolean( bool ) ) {
        b.fail( 'should return a boolean' );
    }
    b.pass( 'benchmark finished' );
    b.end();
});

bench( pkg+'::objects:negative', function benchmark( b ) {
    var values;
    var bool;
    var i;

    values = [
        [],
        {},
        function noop() {}
    ];

    b.tic();
    for ( i = 0; i < b.iterations; i++ ) {
        bool = isString( values[ i % values.length ] );
        if ( typeof bool !== 'boolean' ) {
            b.fail( 'should return a boolean' );
        }
    }
    b.toc();
    if ( !isBoolean( bool ) ) {
        b.fail( 'should return a boolean' );
    }
    b.pass( 'benchmark finished' );
    b.end();
});

[kgryte]

@performant23 Similar to https://github.com/stdlib-js/stdlib/blob/bb104f6a5138d600a4d1460af5380cb4960481d3/lib/node_modules/%40stdlib/assert/is-complex-like/benchmark/benchmark.js#L33, prefer instead

bench( pkg+'::primitives,true', function benchmark( b ) {
    var values;
    var bool;
    var i;

    values = [
        '5',
        'beep'
    ];

    b.tic();
    for ( i = 0; i < b.iterations; i++ ) {
        bool = isString( values[ i % values.length ] );
        if ( typeof bool !== 'boolean' ) {
            b.fail( 'should return a boolean' );
        }
    }
    b.toc();
    if ( !isBoolean( bool ) ) {
        b.fail( 'should return a boolean' );
    }
    b.pass( 'benchmark finished' );
    b.end();
});

as the benchmark description naming convention.

[performant23]

Got it, thanks!

[soumajit23]

hey @kgryte, i feel like i have got a good understanding of the issue and would like to work on a specific package.

[Snehil-Shah]

@Planeshifter I believe this issue got wrongly closed due to the merge