cage-challenge / cage-challenge-4

The TTCP CAGE Challenges are a series of public challenges instigated to foster the development of autonomous cyber defensive agents. This CAGE Challenge 4 (CC4) returns to a defence industry enterprise environment, and introduces a Multi-Agent Reinforcement Learning (MARL) scenario.
https://cage-challenge.github.io/cage-challenge-4/
Other
41 stars 10 forks source link

Updated comms policy handling in observation to match documentation #21

Closed rykrr closed 8 months ago

rykrr commented 8 months ago

This PR updates the communication policy subvector in the flat observation to match the documentation in regards to the values (policy subvector should ideally match the "blocked" subvector) and which subnets appear as blocked (namely the contractor subnet).

I have hardcoded the expected policy matrices into the tests and have looked it over a number of times, but a fresh set of eyes would be appreciated. The order of the elements in the hardcoded matrix matches the table in the documentation.

Note: This update adds an assumption where all three subnets/zones within the HQ Network are treated the same.

Closes #19.

MitchellKiely commented 8 months ago

@rykrr, Wow, those matrices were tiresome to keep track of, I can't imagine how annoying that would have been to make hahah. There was one thing that stood out to me, not sure if this is a documentation error or a programming error though. As per Mission Phase A table, Restricted Zone A disconnects from the internet such that the '1' should be a '0'. Reading the commit, I believe no edges are removed from Restricted Zone A to the internet during this phase. In the tests as well, this is a '1' in the matrices. Likewise, for Mission Phase B, Restricted Zone B should also disconnect from the internet.

From your perspective, should the documentation change or the underlying code? Not sure if this will cause any unwanted errors.

rykrr commented 8 months ago

Good catch, that is indeed a programming error! I'll push a fix shortly. Thanks!

MitchellKiely commented 8 months ago

Looks all good to me, great work! :)