test262 style conformance, etc

[lars-t-hansen]

cc @bterlson

There are many tests in our test262 subdirectory, but these tests don't follow the style of the official test262 repo. Here are some todo items for fixing that.

We need tests for:

• SharedArrayBuffer
• Atomics
• TypedArray changes. Tests go into the TypedArray dir, in the yaml prologue there is a features tag that indictates that the test requires support for SAB. This will likely require copying the test, which is a little nasty.
• DataView changes. Ditto.
• ArrayBuffer changes. Ditto.

Split tests along different dimension:

Consider:

var int_views = [Int8Array, Uint8Array, Int16Array, Uint16Array, Int32Array, Uint32Array];
for ( var idx=0 ; idx < int_views.length ; idx++ ) {
    var View = int_views[idx];
    var view = new View(sab, 32, 20);
...
    assert.sameValue(Atomics.add(view, 8, 10), 0);  // 0 + 10 -> 10
    assert.sameValue(Atomics.sub(view, 8, 5), 10);  // 10 - 5 -> 5

Here, it is OK to keep the loop over the views, but add and sub tests should be moved to separate files.

We have a ditto case for "bad" arrays (not shared memory). They would be in different files, too - the "add" test for good arrays and the ditto for bad arrays would be separated.

(New subdir Atomics; new subdirs of that per operation.)

Test types of properties

Not sure what I have here, but there should be tests that typeof Atomics.add == function, etc. Consider https://github.com/tc39/test262/blob/master/test/built-ins/TypedArray/prototype/every/length.js as a template.

Test many wrong input values

I think we have a good number of these but really, we want to be comprehensive. Looking to atomics-on-other-stuff.js, @anba thinks it's ok for each single file to test all the wrong input types in that array (probably should be broken out as a common function or file for all these tests?) but that each operation, eg Atomics.add, needs to be in a file by itself.

PR on test262

What the subject says, move it out of our directory.

Agents

Probably add something like my 'AgentTest' proposal to the '$' object. @anba suggests we should really try to run these tests in the shell, so this puts some pressure on Apple and Microsoft to provide some kind of worker mechanism that can be used by AgentTest. (For AgentTest details see test262/agenttest_spidermonkey.js in this repo.)

Another question is how to set the agent tests up. The examples I have use sleep() liberally to make sure problems are detectable but this makes them slow. That may be OK. We may also wish to mark the tests as slow, to allow them not to be run. There's also the question of whether using sleep is at all reasonable for all environments, esp the browser environment.

Libraries

It may be useful to have more libraries / utilities, like there is "sleep" and the send/receive string utils now, eg, simple (reliable) locks and barriers.

Want to make a file that provides a function that returns the array of all views that can take atomics (for example).

Style

There are old tests and new tests. Try to emulate the new ones, it makes people happier.

Odds and ends

I have a bunch of tests for SpiderMonkey that may be worth porting even if they are really weird...

[lars-t-hansen]

Work to convert the existing tests to test262 format is ongoing in https://github.com/lars-t-hansen/test262 on the branch "SAB".

[bterlson]

Atomics tests should go in test/built-ins/Atomics/* probably.

I agree we want to enable shell testing here - I'm pushing for adding worker functionality to ch.exe, hopefully others either already have it available or are working on adding it. Anything we add now will probably change in order to run uniformly across hosts (eg. probably I'd go with $.createAgent() that returns an object that represents a new agent).

Libraries you can put into the harness directory and include them with your includes frontmatter. 

Happy to answer any other questions if you point me to them. /cc @syg

[jfbastien]

I'd appreciate having a short description of the proposed shell API, so that we can also run these tests.

[lars-t-hansen]

@jfbastien, I discussed a possible worker API for the shell with Filip (and MS and Google) some time back, I'll loop you into the thread. Happy to discuss alternatives, but let's do so in email.

[lars-t-hansen]

I've started populating the branch I mentioned above: new subdirectories test/built-ins/Atomics and test/built-ins/SharedArrayBuffer.

Atomics tests have been ported over from the proposal repo's test suite.

SharedArrayBuffer tests have (so far) been mostly duplicated from the ArrayBuffer tests.

So far no tests for subtle TypedArray or DataView changes, or for agents.

@anba, @bterlson, @syg - feedback welcome on style & substance of the tests that have landed so far.

[syg]

I opened a PR against test262 with this batch of tests here: https://github.com/tc39/test262/pull/839

[lars-t-hansen]

Considered closed, as discussion continues in the test262 and ecma262 repos.