search

dstl / Stone-Soup

A software project to provide the target tracking community with a framework for the development and testing of tracking algorithms.
https://stonesoup.rtfd.io
MIT License
384 stars 126 forks source link

Monte Carlo Tree Search Sensor Manager and Autonomous STE Examples #1023

Closed timothy-glover closed 1 week ago

timothy-glover commented 1 month ago

This pull request introduces two new sensor management techniques to Stone Soup: MonteCarloTreeSearchSensorManager and MCTSRolloutSensorManager. These two managers are implementations of Monte Carlo tree search (MCTS) that construct a search tree based on actions and estimated future target states and estimate the action value resulting from this. Introducing these managers has also required a modification to reward functions that are to be compatible with MCTS. The ExpectedKLDivergence has been modified to include an extra property called return_tracks that allows the predicted tracks that have been used to calculate the KLD value to be returned to the invoking sensor manager so that it can be stored, in the case of MCTS, in the search tree. ExpectedKLDivergence is currently the only modified reward function. Another modification to ExpectedKLDivergence has also been introduced to allow data association to be performed when predicting tracks that have received multiple detections if sequential state updates are not the desired process. Tests for each of these new features and modifications have also been included in the PR.

Two examples have also been added to the documentation. The first covers the work done for the Loughborough University Stone Soup Kitchen for autonomous STE. This example covers particle filter setup for STE and actionable platform sensor management using BruteForceSensorManager and MultiUpdateExpectedKLDivergence.

The second example is includes an MCTS approach to STE and compares it with a Myopic alternative. The example focusses on more on how to implement a MCTS sensor manager rather than the STE problem that is better covered by the first.

@hpritchett-dstl @nperree-dstl @sdhiscocks

codecov[bot] commented 1 month ago

Codecov Report

Attention: Patch coverage is 93.29609% with 12 lines in your changes missing coverage. Please review.

Project coverage is 93.95%. Comparing base (ab6af2b) to head (1a793fc). Report is 69 commits behind head on main.

Files Patch % Lines
stonesoup/sensormanager/tree_search.py 94.21% 6 Missing and 1 partial :warning:
stonesoup/sensormanager/reward.py 88.88% 3 Missing and 2 partials :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1023 +/- ## ========================================== + Coverage 93.60% 93.95% +0.35% ========================================== Files 202 203 +1 Lines 12990 13133 +143 Branches 2651 2680 +29 ========================================== + Hits 12159 12339 +180 + Misses 588 543 -45 - Partials 243 251 +8 ``` | [Flag](https://app.codecov.io/gh/dstl/Stone-Soup/pull/1023/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dstl) | Coverage Δ | | |---|---|---| | [integration](https://app.codecov.io/gh/dstl/Stone-Soup/pull/1023/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dstl) | `67.99% <93.29%> (+1.72%)` | :arrow_up: | | [unittests](https://app.codecov.io/gh/dstl/Stone-Soup/pull/1023/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dstl) | `88.50% <29.05%> (-0.72%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dstl#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

timothy-glover commented 1 month ago

I'm not sure why the code coverage of tree_search.py and reward.py are so low. On my PC the code coverage of those files are 99% and 96% respectively.