search

hail-is / hail

Cloud-native genomic dataframes and batch computing
https://hail.is
MIT License
983 stars 246 forks source link

[query] the compiler should have tests at the level of Matrix, Table, CDA, and Value IR #13638

Open danking opened 1 year ago

danking commented 1 year ago

What happened?

Currently, almost all of our tests are integration tests which require:

  1. Compiling Scala code.
  2. Building a JAR (takes ~30 seconds on my MBP)
  3. Running pytest (can take as long as 20 seconds)

All of this is a lot slower than iterating with a live running Scala process. We should have tests of various parts of the compiler operating at the IR level. For example, MatrixIR to TableIR lowering should have plenty of in Scala IR-level tests. Likewise for TableIR to CDAIR. The optimizer/simplifier should also have tests at each level which assert certain kinds of code is sufficiently cleaned up by the optimizer.

Version

0.2.122

Relevant log output

No response

danking commented 1 year ago

cc: @patrick-schultz @ehigham, I wrote this in response to our team meeting discussion.