TypeScript Compiler Testing Support

[alexisthedev]

TypeScript + Hephaestus

This pull request consists of the required modules and additions to test the TypeScript compiler with Hephaestus.

Important additions include:

AST-to-TypeScript Translator Core API extension to allow generation of programs with native types of any language (eg. TypeScript's unique union types) Null and Undefined Types Null Constant Literal Types (Native TypeScript) Type Aliases (Native TypeScript) Union Types (Native TypeScript) A more detailed overview is available here.

How to use Hephaestus to test the compiler of TypeScript

Here is a typical command used to test Typescript with Hephaestus.

python3 hephaestus.py -t 0 -w 8 -i 150 --batch 30 -P --language typescript --disable-use-site-variance

Key differences from testing other languages:

• The transformations flag must be set to 0 (-t 0 ) because transformations are not currently supported for TypeScript.
• Must include -P flag, so Hephaestus only runs correctness-preserving transformations (which are zero).
• Must include -disable-use-site-variance because TypeScript doesn't support use-site variance.

Afterward, it is the typical Hephaestus process! (Check in the bugs folder to see the reported faults).

Expected result after running the above command:

Ubuntu 22.04 LTS

Had there been an error, a Main.ts file would have been stored at bugs/{name}/generator/iter_{i} (in the previous case the name is w0zy2).

Ignoring certain error messages

When testing TypeScript, it can be helpful to instruct Hephaestus to ignore certain error messages when they come up. For example, in the above image, the comparison ((77.79) !== (70.238)) has resulted in an error:

ts(2367): This condition will always return 'true' since the types '77.79' and '70.238' have no overlap.

This is working as intended (these two number literals are considered literal types in TypeScript), however knowing that this error might come up in a lot of our programs with TypeScript, we might want to tell Hephaestus to ignore this error message and not report the test program as failed.

For this, we use the flag --error-filter-patterns FILE, where each line in FILE contains a regex pattern that matches the error message we'd like to ignore.

For example in my own patterns file, I have the regex .*This condition will always return.* to ignore the above error.

[alexisthedev]

This has been reopened (as explained here).

Todo list from comments on previous PR #46

• [x] Remove GSOC.md
• [x] Wrap reinstalling the nightly TS build in an install_nightly function
• [x] Add example instructions and expected results in PR description

[alexisthedev]

@StefanosChaliasos The conflicts mentioned do not appear on my end (probably because they are rebase conflicts and not merge conflicts).