search

v8 / v8.dev

The source code of v8.dev, the official website of the V8 project.
https://v8.dev/
Apache License 2.0
953 stars 320 forks source link

How to update test262.status via run-tests.py ? #776

Closed Jxck closed 2 months ago

Jxck commented 2 months ago

Hi, I'm new to v8 contribution.

I'm reading through Testing documentation and running test262 successfully. And now, I wanna update test262 to recent commits, and update test262.status expectation file, according to the docs.

https://v8.dev/docs/test#updating-the-inspector-test-expectations

  1. update DEPS revision
  2. gclient sync
  3. gm x64.release test262

It seems succeeded. and then.

$./tools/run-tests.py \
   test262 \
   --regenerate-expected-files \
   --outdir=out/x64.release

It doesn't update test/test262/test262.status as I expected. What am I missing here ?

I hope sending PR for here once I solve this problem.

LeszekSwirski commented 2 months ago

You're following instructions for regenerating inspector test expectations (which are anyway textual stdout expectations, not pass/fail expectations in .status files). You shouldn't need to update test262.status unless you're making a failing test pass, in which case you can just modify it manually.

Jxck commented 2 months ago

you can just modify it manually

OK, Thanks. (I imagined it can be updated like tools/run_web_tests.py -t Default --reset-results way in chromium too).

LeszekSwirski commented 2 months ago

Basically, the .status files are for exceptions, not general expectations, so we don't expect to need to reset/regenerate them except for exceptional cases.

Jxck commented 2 months ago

OK, let me explain what I wanna do now.

I'm working on adding RegExp.escape function to v8, and test262 has been merged already. When I update test262 up to date in v8, it fails not only RegExp.escape test, but also other added test which v8 doesn't cover now.

I'll pass RegExp.escape in my Patch Set, but wanna ignore (all as fail ?) other test cases. I imagined that test262.status is the files for test262's fail/success expectation, but I found it's false. So, how can I manage test262 result in v8 in this case ?

LeszekSwirski commented 2 months ago

In principle the test262 repository is automatically rolled into V8, so you shouldn't need to manually update it. I pinged the owner of the latest roll (https://chromium-review.googlesource.com/c/v8/v8/+/5756209), you should be able to rebase on top of that and un-skip the Regexp.escape tests.

Jxck commented 2 months ago

Ah I got it, thanks !