search

dstl / Stone-Soup

A software project to provide the target tracking community with a framework for the development and testing of tracking algorithms.
https://stonesoup.rtfd.io
MIT License
388 stars 128 forks source link

Reinforcement Learning Sensor Manager #760

Closed sgboakes closed 4 months ago

sgboakes commented 1 year ago

Added a reinforcement learning sensor manager. Currently only working for Linux-based OSes.

There was a recent numpy update that might make tests fail until tensorflow is updated to fix the error.

nperree-dstl commented 1 year ago

It might also be worth adding some comments on how quickly it runs in comparison the the brute force approach? I ran it as a notebook with %%time in cells 12, 13 and 15 - although the training took 2min 36s, once trained the sensor management tracking loop took 2.38s as opposed to 1min 20s for the brute force

sgboakes commented 1 year ago

The 3.11 test and docs failing are due to pygame (part of tf-agents) not currently supporting Python 3.11. The 3.7 test looks like a tensorflow version error, but unsure how that would be fixed at the moment.

nperree-dstl commented 1 year ago

Everything works well and I'm ready to approve except I'm still struggling to generate the docs without installing the reinforcement dependencies. I wonder if I'm having a similar issue as mentioned here: https://github.com/dstl/Stone-Soup/pull/264#issuecomment-653587283

I'll ask @sdhiscocks about it as I can't get that solution to work yet either.

sgboakes commented 1 year ago

Everything works well and I'm ready to approve except I'm still struggling to generate the docs without installing the reinforcement dependencies. I wonder if I'm having a similar issue as mentioned here: #264 (comment)

I'll ask @sdhiscocks about it as I can't get that solution to work yet either.

I've been able to get the docs running without installing reinforcement dependencies. With a fresh virtual environment, I clone the pull request, install the dev tools and ensure that nothing remains in the docs source folder:

pip install -e ".[dev,orbital]" git clean -xf docs/source/auto_*

Then I do a git pull to download the notebook, rst, and md5 files from the PR repo, and then build the docs, and that works for me.

In the comment you linked, md5 checksums should now be OS independent, so it should work on both windows and linux. I verified the above method on Windows using WSL.