slim version of counterfactual predictions for frontend use

[rfl-urbaniak]

using ChiRho, Pyro, and pyro.infer.Predictive in the front to evaluate counterfactual claims end was too demanding for front-end use. This PR provides a slim version that can be used in the front-end without even having Pyro or ChiRho among the dependencies. This is possible as for now we are interested in queries of one format, answers to which can be calculated pretty much by hand once we have cached model samples. The caching can be done once after fresh model training and needs not to be repeated unless model guides and params are modified.

1. setup.py now divides dependencies between what's needed for development and a minimal setup that should be sufficient on the front end. The latter does not include Pyro or ChiRho, in contrast with the former.

• To install the slim version: pip install -e .
• To install the dev version: pip install -e .[dev] or pip install -e ".dev" (if the former doesn't work) This was needed so that the slim version can be decoupled from heavy dependencies. It should be sufficient for front end use.

Note: pytests won't work as some tests involve heavier dependencies. but you can install an intermediate version:

pip install -e ".[test]"

which should make make test_notebooks work properly (but not make test). The former is a backend dev proxy for whether all basic functionalities needed for the front end are in good shape.

2. For the above reason, notebook tests have been decoupled from the standard pytest test. They now check the running of all notebooks in docs/guides and are triggered by make test_notebooks. In contrast, make test triggers first the pytest test and the notebook dry runs. This was needed as all notebooks were repeatedly run to test the slim version in development.

3. The data folder now contains a folder tau_samples. This was generated using cities.modeling.tau_caching_pipeline. All front end functionalities should be available if this folder is present, even without data/model_guides, which is a much heavier folder - try deleting the latter on the server, let me know if anything fails.

4. CausalInsight has been expanded. generate_tensed_samples serializes tau samples in the corresponding data folder if such samples for a particular intervention-outcome pair are missing. It also contains a new method, get_tau_samples, which is a slim counterpart of generate_tensed_samples and should be used instead in front end applications.

5. In dev mode, inspect 'docs/experimental_notebooks/slimming/slim_insight.ipynb` which illustrates performance differences between the two methods.

6. In slim mode visit docs/experimental_notebooks/slimming?slim_use.ipynb, which shows the pieces of code that should replace what's currently used in the front end. The change is rather minimal. Make sure it works in the slim mode.

7. In the [test] mode (but not dev) run make test notebooks to ensure all front end functionalities are still running.

8. Also, make format and make lint also cleans up the notebooks in docs/guides.

[rfl-urbaniak]

I now see merge conflicts, @riadas did you make any changes in the target? If any controversial conflict I'll deal with it tomorrow, jus take a look if you have the time and let me know.

[riadas]

Works on the frontend without out-of-memory errors! Thanks for getting on this so quickly.

[riadas]

Works on the frontend without out-of-memory errors! Thanks for getting on this so quickly.