search

facebook / flow

Adds static typing to JavaScript to improve developer productivity and code quality.
https://flow.org/
MIT License
22.09k stars 1.86k forks source link

Feedback request: use node tooling for checking Test262 tests? #5247

Open leobalter opened 7 years ago

leobalter commented 7 years ago

Hi!

Right now there is a work in progress for Prepack to roll a better integration of Test262 test files and a configurable filters list. You can find this work here: https://github.com/facebook/prepack/pull/1122

Due to the current nature of Flow parsing Test262 tests, I wonder if a similar node-based tooling script would be interesting to add, as the current scripts in this project already iterate through all the files and verify that each test can be parsed by Flow.

This work can be done by using https://github.com/bocoup/test262-stream or https://github.com/bocoup/test262-integrator (to apply any necessary filters).

@mroch, we've been in contact for the last couple months. I'm willing to do the work, but not until I have your feedback and explicit "go ahead".

mroch commented 7 years ago

so, I started out using node to run these, which requires shelling out to flow for each one. child_process.exec has shell overhead, so I also tried spawn. it took something like 11 minutes. using spawn via cluster on 32 CPUs (!!!) still took around a minute. doing it serially in a single process in parse_test262.ml takes like 15 seconds.

I would love to reuse test262-stream, especially for more complicated test workflows. for example, we want to print the AST and actually run the tests (vs just parsing them), and I think it'd be great for that since that necessarily involves node as well.

so in theory I agree, but I'd be curious/concerned about perf. on the other hand, perf's not a dealbreaker, especially if filtering makes it possible to only run the relevant tests when trying to fix something.

leobalter commented 7 years ago

Nice! This gives some good thinking to proceed.

I can tell you I performance is not really something I can achieve with the test262-stream as you do in 15 seconds - that's impressive, btw.

I would love to reuse test262-stream, especially for more complicated test workflows. for example, we want to print the AST and actually run the tests (vs just parsing them), and I think it'd be great for that since that necessarily involves node as well.

I want to give it a try. I'll keep you posted.