cage-challenge / cage-challenge-4

The TTCP CAGE Challenges are a series of public challenges instigated to foster the development of autonomous cyber defensive agents. This CAGE Challenge 4 (CC4) returns to a defence industry enterprise environment, and introduces a Multi-Agent Reinforcement Learning (MARL) scenario.
https://cage-challenge.github.io/cage-challenge-4/
Other
41 stars 10 forks source link

Challenge 4 BlueEnterpriseWrapper - BlockTrafficZone and AllowTrafficZone don't produce the expected result #18

Closed William-John-Valentine closed 8 months ago

William-John-Valentine commented 8 months ago

After an agent uses the BlockTrafficZone action, I would expect to see a change in the "Blocked Subnets" section(s) of the observation vector. I see no changes.

action.get_params() shows that BlockTrafficZone causes an error to be logged, for example:

image

In CybORG/Simulator/Actions/ConcreteActions/ControlTraffic.py, the execute_control_traffic() methods of BlockTrafficZone and AllowTrafficZone both search state.subnet_name_to_cidr.keys() (the subnet names) to determine if a subnet is valid, however BlueFixedActionWrapper's _populate_action_space() passes the CIDR values for the from_subnet and to_subnet parameters (line 278 of BlueFixedActionWrapper.py).

rykrr commented 8 months ago

Hi @William-John-Valentine,

Thank you very much for the detailed bug report! I have put together a quick fix and updated the tests in rk-fix-wrapper-zonectrl. These changes appear to be compatible with how actions handle blocking, however, I haven't had the chance to fully verify this and I will try my hand at putting together a more comprehensive test once I have a bit more time to spare. In the meantime, would you be willing to verify that this works?

Thank you again,

William-John-Valentine commented 8 months ago

Hi @rykrr No problem, BlockTrafficZone and AllowTrafficZone seem to be working as intended in rk-fix-wrapper-zonectrl. Cheers, William

rykrr commented 8 months ago

Thank you for checking, much appreciated!