First What-If Analysis (Issues #3, #4)

[stefan-grafberger]

Issue #, if available: #3, #4

Description of changes:

• Added interface for what-if analysis
• Implemented a first what-if analysis, DataCorruption. However, it is not optimized for performance yet; that will be a to-do in the future.
• The mechanism to specify the corruptions will need some prettying up in the future.
• Added logging messages to analyze the performance overhead of different steps.
• Adding a first simple demo notebook, but this will also need a lot of polishing later.
• The DataCorruption testing could be a bit more extensive by verifying where corruption nodes are inserted, but this would also require a lot of updates to the tests while this is still in active development.
• Added convenience functions to save images of intermediate DAGs to some file location.
• Currently, they are used extensively in tests; maybe we should stop including them in commits at some point.
• Added capturing for the stdout output in user code.
• Includes a very simple multi-query optimizer that reuses results by generating new unique ids for all nodes that have a changed node as an ancestor and then reusing intermediate results for all nodes that pass equality checks. In the future, we will need to think about more sophisticated strategies.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[stefan-grafberger]

Codecov Report

Merging #7 (4af1c18) into main (4e13bdc) will increase coverage by 0.44%. The diff coverage is 96.55%.

@@            Coverage Diff             @@
##             main       #7      +/-   ##
==========================================
+ Coverage   95.64%   96.09%   +0.44%     
==========================================
  Files          18       22       +4     
  Lines        1861     2174     +313     
==========================================
+ Hits         1780     2089     +309     
- Misses         81       85       +4     

Impacted Files	Coverage Δ
mlwhatif/monkeypatching/_patch_pandas.py	89.37% <ø> (+0.72%)	:arrow_up:
mlwhatif/analysis/_what_if_analysis.py	73.68% <73.68%> (ø)
mlwhatif/_pipeline_analyzer.py	92.98% <92.98%> (ø)
mlwhatif/analysis/_data_corruption.py	96.69% <96.69%> (ø)
mlwhatif/__init__.py	100.00% <100.00%> (ø)
mlwhatif/_inspector_result.py	100.00% <100.00%> (ø)
mlwhatif/analysis/_analysis_utils.py	100.00% <100.00%> (ø)
mlwhatif/execution/_dag_executor.py	95.31% <100.00%> (-0.08%)	:arrow_down:
mlwhatif/execution/_multi_query_optimizer.py	100.00% <100.00%> (ø)
mlwhatif/instrumentation/_dag_node.py	100.00% <100.00%> (ø)
... and 6 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 4e13bdc...4af1c18. Read the comment docs.