Learning Neural Causal Models from Unknown Interventions

This is a Pytorch implementation of the Learning Neural Causal Models from Unknown Interventions paper. Here we learn the causal model based on a meta-learning transfer objective from unknown intervention data. Please cite:

Nan Rosemary Ke*, Olexa Bilaniuk*, Anirudh Goyal, Stefan Bauer, Hugo Larochelle, Chris Pal, Yoshua Bengio

@article{ke2019learning,
    title={Learning Neural Causal Models from Unknown Interventions},
    author={Ke, Nan Rosemary and Bilaniuk, Olexa and Goyal, Anirudh and Bauer, Stefan and Larochelle, Hugo and Pal, Chris and Bengio, Yoshua},
    journal={arXiv preprint arXiv:1910.01075},
    year={2019}
}

Installation

1. This code is based on Pytorch. The conda enviroment for running this code can be installed as follows,

conda env create -f environment.yml

pip install -e .

CPU features required: AVX2, FMA (Intel Haswell+)

2. Training code

# chain3
python run.py train --seed 1 --train_functional 10000 --mopt adam:5e-2,0.9 --gopt adam:5e-3,0.1 -v 500 --lsparse 0.1 --bs 256 --ldag 0.5 --predict 100 --temperature 1 --limit-samples 500 -N 2 -p chain3  

# fork3
python run.py train --seed 1 --train_functional 10000 --mopt adam:5e-2,0.9 --gopt adam:5e-3,0.1 -v 500 --lsparse 0.1 --bs 256 --ldag 0.5 --predict 100 --temperature 1 --limit-samples 500 -N 2 -p fork3

# collider3
python run.py train --seed 1 --train_functional 10000 --mopt adam:5e-2,0.9 --gopt adam:5e-3,0.1 -v 500 --lsparse 0.1 --bs 256 --ldag 0.5 --predict 100 --temperature 1 --limit-samples 500 -N 2 -p collider3

# confounder3
python run.py train --seed 1 --train_functional 10000 --mopt adam:5e-2,0.9 --gopt adam:5e-3,0.1 -v 500 --lsparse 0.1 --bs 256 --ldag 0.5 --predict 100 --temperature 1 --limit-samples 500 -N 2 -p confounder3

--seed specifies the random seed

--mopt specifies the optimizer and learning rate used to train the functional parameters

--gopt specifies the optimizer and learning rate used to train the structural parameters

--predict specifies the number of samples used for predicting the intervened node. Specifying 0 for this argument uses the groundtruth intervention node.

--temperature specifies the temperature setting fot the softmax for the groundtruth structured causal model.

-N specifies the number of categories for the categorical distribution

-M specifies the number of discrete random variables

--graph allows one to specify via the command-line several causal DAG skeletons.

-p specifies, by name, one of several --graph presets for groundtruth causal graphs (e.g. chain3).

--train_functional specifies how many iterations to train the functional parameters.

--limit-samples specifies the number of samples used per intervention. Suggest to use 500 for graphs of size < 10 and 1000 for graphs size between 10 and 15.

By default, the models and log files are stored in the work directory.

3. Extracting useful information from log files.

Strip all the SLURM output of interruptions:

scripts/strip_interrupts.py slurm-%j.out         > STRIPPED_LOGFILE.txt

Extract gamma CE timeseries

scripts/series_gammace.py   STRIPPED_LOGFILE.txt > gammace.txt

Compute AUROC-over-time

scripts/series_auc.py       STRIPPED_LOGFILE_1.txt STRIPPED_LOGFILE_2.txt STRIPPED_LOGFILE_3.txt STRIPPED_LOGFILE_4.txt STRIPPED_LOGFILE_5.txt > auc.txt