search

hephaestus-compiler-project / hephaestus

A framework for testing compilers' type checkers
https://hephaestus-compiler-project.github.io/
GNU General Public License v3.0
49 stars 10 forks source link

Boost the performance of the tool #15

Closed theosotr closed 3 years ago

theosotr commented 3 years ago

The bottleneck of our tool is the time spent on compiler. We can boost the performance of testing if we feed multiple files to compiler. For example, compiling 30 programs at once takes only 15 seconds, while compiling 30 programs one at a time takes roughly 2 minutes.

This is a great improvement. So we need to adapt the tool for supporting compilation of multiple files.

StefanosChaliasos commented 3 years ago

Any change should not break the current CLI. Take into consideration both oracles (CORRECTNESS_PRESERVING=True and CORRECTNESS_PRESERVING=False).

StefanosChaliasos commented 3 years ago

We should also consider adding thread support for the generation and the transformations.

theosotr commented 3 years ago

No, we will use multiprocessing as it is now, as threading fits for I/O bound tasks.

The generation of programs should be decoupled from compilation. The generation and application of transformations will be done through multiprocessing. After generating and applying transformation to a user-specified number of programs (i.e., 30-50), we will invoke the compiler for testing these programs. Then, we will repeat the process, i.e., we will generate another bunch of programs.

theosotr commented 3 years ago

We can generate and apply transformations to around 200 programs in 2 minutes (with 8 workers).

StefanosChaliasos commented 3 years ago

We should create a new option for this. Probably --batch NUMBER_OF_PROGRAMS_PER_BATCH, and then we will specify both iterations and batch. It could be implemented simply as a method in Executor. Maybe we can decouple transformations from Executor.