Lit marks tests as XFAIL by generating a list of discovered_tests and marking each of them as XFAIL if their relative filenames or fullnames appear in its opts.xfail which is populated from either the --xfail-tests flag or LIT_XFAIL env var before the tests actually run. When gtest sharding is enabled, every test's name in discovered_tests initially contains a numeric suffix like FooTest\1\2 before the tests runs, so the test names don't match the names the User passes to the arguments above. The googletest module will replace these numbers with the actual test suite and case names by reading them from gtest json files only after the tests are finished running. This means that it's impossible to mark a test as XFAIL using the above mechanisms when gtest sharding is enabled. One can only use the Python test decorators, or the shell test XFAIL: headers, or any of the other source-code based mechanisms.
kendalharland
commented
1 month ago
Lit marks tests as XFAIL by generating a list of
discovered_testsand marking each of them as XFAIL if their relative filenames or fullnames appear in itsopts.xfailwhich is populated from either the--xfail-testsflag orLIT_XFAILenv var before the tests actually run. When gtest sharding is enabled, every test's name indiscovered_testsinitially contains a numeric suffix likeFooTest\1\2before the tests runs, so the test names don't match the names the User passes to the arguments above. Thegoogletestmodule will replace these numbers with the actual test suite and case names by reading them from gtest json files only after the tests are finished running. This means that it's impossible to mark a test as XFAIL using the above mechanisms when gtest sharding is enabled. One can only use the Python test decorators, or the shell testXFAIL:headers, or any of the other source-code based mechanisms.