Problems running benchmarks (crashing, too slow)

[lahma]

We have our comparison benchmarks at Jint repo and it's getting a bit tedious to try to run the benchmarks as yantra is either really slow (regex) or just crashing (linqjs), is there something that could be done? I tried this run with NET 8.0 RC 2 to ensure there's nothing legacy hindering the run.

We are using the (current latest) version 1.2.201.

BenchmarkDotNet v0.13.10, Windows 11 (10.0.23580.1000)
AMD Ryzen 9 5950X, 1 CPU, 32 logical and 16 physical cores
.NET SDK 8.0.100-rc.2.23502.2
  [Host]     : .NET 8.0.0 (8.0.23.47906), X64 RyuJIT AVX2
  DefaultJob : .NET 8.0.0 (8.0.23.47906), X64 RyuJIT AVX2

Method	FileName	Mean	StdDev	Median	Rank	Allocated
NilJS	array-stress	6,262.843 μs	7.8334 μs	6,261.519 μs	1	4533.76 KB
Jint_ParsedScript	array-stress	6,652.716 μs	12.6513 μs	6,650.304 μs	2	7090.82 KB
Jint	array-stress	6,756.168 μs	26.7944 μs	6,750.390 μs	3	7112 KB
Jurassic	array-stress	11,105.377 μs	15.1026 μs	11,102.109 μs	4	11647.14 KB
YantraJS	array-stress	72,455.529 μs	9,736.4723 μs	71,517.194 μs	5	255641.74 KB
YantraJS	dromaeo-3d-cube	NA	NA	NA	?	NA
NilJS	dromaeo-3d-cube	8,108.019 μs	41.8822 μs	8,107.017 μs	1	4693.22 KB
Jint_ParsedScript	dromaeo-3d-cube	22,872.079 μs	14.2433 μs	22,871.044 μs	2	5934.51 KB
Jint	dromaeo-3d-cube	23,734.911 μs	73.2514 μs	23,754.294 μs	3	6191.82 KB
Jurassic	dromaeo-3d-cube	53,814.428 μs	259.2690 μs	53,760.050 μs	4	10668.95 KB
NilJS	dromaeo-core-eval	1,568.331 μs	4.0252 μs	1,568.139 μs	1	1598.62 KB
Jint	dromaeo-core-eval	3,497.220 μs	13.6809 μs	3,503.786 μs	2	350.84 KB
Jint_ParsedScript	dromaeo-core-eval	3,816.991 μs	21.4332 μs	3,819.798 μs	3	333.79 KB
YantraJS	dromaeo-core-eval	9,528.317 μs	53.0721 μs	9,543.352 μs	4	37062.2 KB
Jurassic	dromaeo-core-eval	10,120.734 μs	21.7763 μs	10,118.962 μs	5	2883.97 KB
Jint_ParsedScript	dromaeo-object-array	40,470.678 μs	44.7242 μs	40,477.631 μs	1	100753.17 KB
Jint	dromaeo-object-array	42,959.619 μs	53.1244 μs	42,970.792 μs	2	100793.58 KB
Jurassic	dromaeo-object-array	44,016.303 μs	170.5113 μs	43,995.379 μs	3	25812.64 KB
NilJS	dromaeo-object-array	69,170.482 μs	163.0611 μs	69,178.200 μs	4	17697.98 KB
YantraJS	dromaeo-object-array	860,585.140 μs	127,296.6544 μs	819,388.950 μs	5	4011710.62 KB
YantraJS	droma(...)egexp [21]	NA	NA	NA	?	NA
Jint	droma(...)egexp [21]	158,868.937 μs	3,933.9390 μs	156,512.550 μs	1	167358.62 KB
Jint_ParsedScript	droma(...)egexp [21]	162,927.586 μs	5,089.9583 μs	161,303.350 μs	2	169166.13 KB
NilJS	droma(...)egexp [21]	676,215.607 μs	11,930.9836 μs	678,765.550 μs	3	765846.57 KB
Jurassic	droma(...)egexp [21]	746,804.217 μs	18,786.0370 μs	752,323.100 μs	4	829618.68 KB
Jint_ParsedScript	droma(...)tring [21]	254,910.140 μs	9,116.6810 μs	259,280.450 μs	1	1322149.99 KB
Jint	droma(...)tring [21]	262,166.848 μs	12,284.3740 μs	259,798.350 μs	1	1322422.89 KB
NilJS	droma(...)tring [21]	280,793.741 μs	10,606.6209 μs	276,202.000 μs	2	1377934.8 KB
Jurassic	droma(...)tring [21]	298,070.185 μs	6,793.6463 μs	296,046.350 μs	3	1458278.95 KB
YantraJS	droma(...)tring [21]	10,731,120.969 μs	431,483.3871 μs	10,686,728.800 μs	4	49977816.79 KB
NilJS	droma(...)ase64 [21]	33,989.818 μs	146.7131 μs	33,993.293 μs	1	19604.35 KB
Jint	droma(...)ase64 [21]	48,431.947 μs	109.4991 μs	48,427.491 μs	2	6806.59 KB
Jint_ParsedScript	droma(...)ase64 [21]	49,570.704 μs	107.4231 μs	49,546.318 μs	3	6720.88 KB
Jurassic	droma(...)ase64 [21]	76,290.011 μs	216.4848 μs	76,319.671 μs	4	73294.38 KB
YantraJS	droma(...)ase64 [21]	154,618.890 μs	4,555.8418 μs	154,737.138 μs	5	804985.95 KB
Jint_ParsedScript	evaluation	11.187 μs	0.0420 μs	11.189 μs	1	27.47 KB
Jint	evaluation	25.327 μs	0.1767 μs	25.274 μs	2	36.04 KB
NilJS	evaluation	38.699 μs	0.0817 μs	38.687 μs	3	23.47 KB
YantraJS	evaluation	144.559 μs	2.2947 μs	145.059 μs	4	962.86 KB
Jurassic	evaluation	1,408.054 μs	7.6602 μs	1,408.891 μs	5	420.34 KB
YantraJS	linq-js	NA	NA	NA	?	NA
Jint_ParsedScript	linq-js	83.371 μs	0.5199 μs	83.233 μs	1	226.51 KB
Jint	linq-js	1,700.059 μs	39.1810 μs	1,689.233 μs	2	1275.27 KB
NilJS	linq-js	6,624.719 μs	46.8200 μs	6,624.889 μs	3	4121.1 KB
Jurassic	linq-js	34,481.704 μs	92.6770 μs	34,499.567 μs	4	9252.69 KB
Jint_ParsedScript	minimal	2.609 μs	0.0237 μs	2.607 μs	1	12.98 KB
NilJS	minimal	3.867 μs	0.0094 μs	3.869 μs	2	4.81 KB
Jint	minimal	3.972 μs	0.0141 μs	3.975 μs	3	14.37 KB
YantraJS	minimal	137.705 μs	1.4817 μs	138.369 μs	4	957.43 KB
Jurassic	minimal	255.173 μs	0.6002 μs	255.196 μs	5	386.21 KB
YantraJS	stopwatch	85,984.586 μs	802.0424 μs	85,970.117 μs	1	224314.23 KB
NilJS	stopwatch	174,987.804 μs	1,290.5212 μs	174,701.000 μs	2	97360.8 KB
Jurassic	stopwatch	187,415.613 μs	649.4774 μs	187,509.167 μs	3	156936.68 KB
Jint_ParsedScript	stopwatch	273,097.233 μs	1,252.6019 μs	273,326.800 μs	4	53015.63 KB
Jint	stopwatch	273,747.000 μs	713.7972 μs	273,666.700 μs	4	53040.3 KB

Benchmarks with issues: EngineComparisonBenchmark.YantraJS: DefaultJob [FileName=dromaeo-3d-cube] EngineComparisonBenchmark.YantraJS: DefaultJob [FileName=droma(...)egexp [21]] EngineComparisonBenchmark.YantraJS: DefaultJob [FileName=linq-js]

[ackava]

I was able to fix the issue with some crashes, however, linq-js is accessing global this, which isn't available in strict mode as YantraJS treats all scripts as strict only.

So please change the code,

    [Benchmark]
    public void YantraJS()
    {
        var engine = new YantraJS.Core.JSContext();
        engine.Eval(_files[FileName]);
    }

To following,

    [Benchmark]
    public void YantraJS()
    {
        var engine = new YantraJS.Core.JSContext();
        // By default YantraJS is strict mode only, in strict mode
        // we need to pass `this` explicitly in global context
        // if script is expecting global context as `this`
        engine.Eval(_files[FileName], null, engine);
    }

I was able to run all benchmarks successfully with version 1.2.203

[lahma]

Thank you for fast response, I was now able to run all the benchmarks also with YantraJS and the results have been updated.