search

facebookexperimental / object-introspection

Object Introspection (OI) enables on-demand, hierarchical profiling of objects in arbitrary C/C++ programs with no recompilation.
Apache License 2.0
160 stars 13 forks source link

oilgen: add to integration test framework #446

Closed JakeHillion closed 6 months ago

JakeHillion commented 8 months ago

oilgen: add to integration test framework

TODO: Replace the references to local paths.

oilgen (the basis of Ahead Of Time compilation for OIL) has never been passed through our large test suite and has instead had more focused testing and large examples. When consuming DWARF information in a similar fashion to JIT OIL this was okay, but with the new Clang AST based mechanism it means we have very little coverage.

This change adds an oilgen test for every test case that has an oil test. Relying on the build system to create the test target as before would make it difficult to have failing tests, so we move the build into the integration test runner. This involves:

  1. Writing the input source to a file.
  2. Consuming it with oilgen to get the implementation object file.
  3. Compiling the input source and linking it with this file.
  4. Running the newly created target.

This approach can give the full error message at any stage that fails and will fail the test appropriately. The downside is the build system integration is more difficult, as we need the correct compiler flags for the target and to use the correct compiler. It would be very tricky to replicate this in a build system that's not CMake, so we will likely only run these tests in open source.

Test plan:

codecov-commenter commented 8 months ago

Codecov Report

Attention: 83 lines in your changes are missing coverage. Please review.

Comparison is base (beb404e) 44.63% compared to head (5fa72c4) 44.32%.

Files Patch % Lines
test/integration/runner_common.cpp 0.00% 82 Missing :warning:
test/integration/runner_common.h 0.00% 1 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #446 +/- ## ========================================== - Coverage 44.63% 44.32% -0.31% ========================================== Files 121 121 Lines 11905 11988 +83 Branches 1947 1968 +21 ========================================== Hits 5314 5314 - Misses 5738 5821 +83 Partials 853 853 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

JakeHillion commented 6 months ago

Closing for now as CMake not providing linker commands means this can't work reliably on all machines in this form.