Rational OpenCog Controlled Agent, or ROCCA, is a project aiming at creating an opencog agent that acts rationally in OpenAI Gym environments (including Minecraft via MineRL and Malmo).
At its core it relies on PLN (Probabilistic Logic Networks) for both learning and planning. In practice most of the learning is however handled by the pattern miner, which can be seen as a specialized form of PLN reasoning. Planning, the discovery of cognitive schematics, is handled by PLN and its temporal reasoning rule base. Decision is currently a hardwired module, heavily inspired by OpenPsi with a more rational sampling procedure (Thompson Sampling for better exploitation vs exploration tradeoff).
For now learning is able to
The next steps are
OpenCog tools
Third party tools
Python 3.10 offers a better out-of-the-box type annotation system than Python 3.8 and is thus the default required version. However you may still use Python 3.8 by checking out the python-3.8-compatible branch. Beware that such Python 3.8 branch may not be as well maintained as the master.
In the root folder enter the following command (you might need to be root depending on your system):
pip install -e .
For the tools used for development:
pip install -r requirements-dev.txt
An OpencogAgent defined under the rocca/agents
folder is provided
that can used to implement agents for given environments. See the
examples under the examples
folder.
There are Jupyter notebooks provided for experimentation as well. To run them call jupyter notebook on a ipynb file, such as
jupyter notebook 01_cartpole.ipynb
Some experiments, notably the notebooks, use TensorBoard via the
tensorboardX
library to store event files that show certain metrics
over time for training / testing (for now it's just rewards).
By default, event files will be created under the
runs/<datetime><comment>
directory. You can invoke tensorboard --logdir runs
from the project root to start an instance that will
see all the files under that directory. Open your browser to
http://localhost:6006
to see its interface.
If you write code in notebooks that is exported (has the #export
comment on top of the cell), remember to invoke nbdev_build_lib
to
update the library. Remember to use black
for formatting, you can
invoke black .
from the project root to format everything.
You can also use the Makefile for your convenience, invoking make rocca
will do both of the above in sequence.
The .devcontainer
folder has configuration for VS Code
devcontainer
functionality. You can use it to setup a development environment very
quickly and regardless of the OS you use.
vncpassword
. You can use any VNC client to see the results of
rendering Gym environments this way.Using type annotations is highly encouraged. One can type check the entire Python ROCCA code by calling
tests/mypy.sh
from the root folder.
To only type check some subfolder, you may call mypy.sh
from that
subfolder. For instance to type check the examples
subfolder
cd examples
../tests/mypy.sh
Or directly run mypy
on files or directories, such as
mypy rocca/rocca/agents/core.py
tests/mypy.sh
merely calls mypy on all python files in the directory
from which it is called while filtering out some error messages.
Simply run pytest
in the root folder.
There is no ROCCA paper per se yet. In the meantime here is a list of related references