munichpavel
commented
2 years ago Automated testing
Why test?
Risk management! See SRE book.
Dopamine hits
Testing what?
For us, the "application" is a repository with slides and example notebooks that should just work. This means a notebook that does not run given the code in the versioned repository is a bug.
Beware false positives and confidence
No such thing as robust, fully automated testing--have to understand meaning of application, getting 100% test coverage not enough.
Above green check was false positive, as our "application" includes notebooks that should just work, was not the case.
--> Add test that notebooks actually run without error.
Examples
from running notebooks-run test
``console 9 from graphviz import Digraph, Graph 11 # The below are packages I used to solve these exercises, but may be safely removed ---> 12 from causalgraphicalmodels import CausalGraphicalModel 13 from fake_data_for_learning import BayesianNodeRV, SampleValue, FakeDataBayesianNetwork
ModuleNotFoundError: No module named 'causalgraphicalmodels' ModuleNotFoundError: No module named 'causalgraphicalmodels'
```console
import os
from pathlib import Path
import numpy as np
import pandas as pd
# Only needed to generate graphs, may be safely ommitted
# once you comment out relevant cells below
from graphviz import Digraph, Graph
# The below are packages I used to solve these exercises, but may be safely removed
from causalgraphicalmodels import CausalGraphicalModel
from fake_data_for_learning import BayesianNodeRV, SampleValue, FakeDataBayesianNetwork
------------------
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
Input In [1], in <cell line: 13>()
11 # The below are packages I used to solve these exercises, but may be safely removed
12 from causalgraphicalmodels import CausalGraphicalModel
---> 13 from fake_data_for_learning import BayesianNodeRV, SampleValue, FakeDataBayesianNetwork
ImportError: cannot import name 'BayesianNodeRV' from 'fake_data_for_learning' (/Users/pauldev/.virtualenvs/rl/lib/python3.9/site-packages/fake_data_for_learning/__init__.py)
ImportError: cannot import name 'BayesianNodeRV' from 'fake_data_for_learning' (/Users/pauldev/.virtualenvs/rl/lib/python3.9/site-packages/fake_data_for_learning/__init__.py)Other benefits
Risk managment: can make changes / improvements with confidence.
e.g. removing unneeded dependencies that slow down build
Psychology aspect: keep your amygdala calm, and your creative juices flowing.
Pipeline TODOs
Desiderata
Resources