1. Introduction

The project's main aim is to support a Bayesian framework for reinforcement learning. For this reason, most of the documentation is generated via Doxygen.

1.1. File organisation

Directories:

dat/ - data dat/r1e7.bin - random bits, useful for consistent PSR runs

src/ - source code

src/algorithms/ - algorithms for reinforcement learning src/core/ - miscellaneous routines used everywhere else src/environments/ - test benches for algorithms src/geometry/ - anything pertaining to geometry goes here src/models/ - models of processes, used by algorithms src/statistics/ - RNGs, distributions, used by models or algorithms src/tests/ - test code

An algorithm in algorithms/ may use classes from the models/ directory. Models may use statistics or algorithms for implementation.

Environments can have the ability to create an analytical MDP or POMDP model (if not, an approximate one could be built via simulation perhaps).

Currently, statistical models of MDPs are included in src/models/ rather than in src/statistics/, although the remains of an earlier effort are there.

Algorithms for reinforcement learning are arleady implemented in src/algorithms/

Most of the top-level src/ directories also have a subdirectory called tests/ which is used for three purposes:

   a) regression/unit testing
   b) experimental test benches
   c) prototyping new algorithms.

2. Compilation

2.1. Requirements

GSL (GNU Scientific Library), including development packages. LAPACK, including development packages. ATLAS, including development packages.

You can do that via the line:

sudo apt-get install libgsl0-dev libgsl0ldbl libatlas3gf-base libatlas-dev libatlas-base-dev libblas-dev libatlas3gf liblapack-dev liblapack3gf

2.2. Compiling the library

In the top directory do:

a) Choose your compiler:

For GCC: cp src/Make-gcc.mk src/Make-default.mk

For LLVM/CLANG: cp src/Make-clang.mk src/Make-default.mk

(b) make clean

(c) make

2.3. Compiling test programs

Tests and example programs are available all directories named:

src/*/tests

To compile an example program called src/foo/tests/blah.cc you need to cd to src/foo/tests/ and type 'make blah'.

EXAMPLE

cd src/algorithms/tests make online_algorithms ./bin/online_algorithms

3. Generating API documentation

At the top level directory, type:

doxygen

4. TODO

4.1. implement rl-glue backend 4.2. improve documentation (ongoing) 4.3. remove marginally useful code (ongoing) 4.4. replace all matrix / vector routines with GSL/BLAS (in progress)

test