Installation:

To install please create an environment

conda create -n qreps python=3.7

Then activate the environment

conda activate qreps

Install the library locally in edit mode

pip install -e.[experiments]

Figure 1:

To run the experiments move to each folder and runner the runner files. To reproduce the experiments of Figure 1 run

python exps/logistic_vs_squared_bellman_error.py

Figure 2:

To run the experiments of Figure 2 move to the environments folder

cd exps/environments

and from there launch all experiments:

python launch_experiments.py 

To merge the results of the different random seeds run:

python merge_results.py $ENV_NAME

and replace on ENV_NAME the environment you want to merge. Finally, run the plotter

python plot_all.py

Note:

If this takes too long and rather just run a single random seed per environment, please move to each environment folder. Within each folder run the runner and the plotter. For example, for the two state stochastic environment do:

cd exps/environments/two_state_stochastic

and then run

python two_state_stochastic_run.py

to reproduce the experiment. To plot just run

python two_state_stochastic_plot.py

Figure 4.

To reproduce the action_gap experiments, go to:

cd exps/action_gap

and then run

python action_gap_run.py

to reproduce the experiment. To plot just run

python action_gap_plot.py

Figure 5.

To reproduce the bias experiments, go to:

cd exps/bias

and then run

python bias_run.py

to reproduce the experiment. To plot just run

python bias_plot.py