Open bnoordhuis opened 4 hours ago
Regarding the alphabetical order, the list of test files should already be sorted in a deterministic order:
/* find js files from the directory tree and sort the list */
static void enumerate_tests(const char *path)
{
namelist_t *lp = &test_list;
int start = lp->count;
ftw(path, add_test_file, 100);
qsort(lp->array + start, lp->count - start, sizeof(*lp->array),
namelist_cmp_indirect);
}
If we run multiple threads, the errors will be written to test262_errors.txt in a non deterministic order (I guess this is what you meant @bnoordhuis ) so the file should be re-sorted after all threads complete, which is straightforward if all errors are written as single lines. Storing these errors in one of more lists (eg: in the test list) and writing them at the end is a palatable approach. The file test262_report.txt does not need sorting and can be used in case run-test262 crashes.
We run an ever larger slice of test262 and running time grows concomitantly. Using threads should give roughly an N-fold speedup.
Caveat: failing tests should be written in a reproducible manner to test262_errors.txt; for instance, by sorting them alphabetically.