Next steps - Githubissues

williamshen-nz commented 2 years ago

[x] Reward for Naive - naive implemented
- just need to plot against frequency of updates and how long we wait. (not really for now)
[x] Same for random (not really useful so won't bother)
[x] Added support for num steps before simulation and num steps per action in C++ code
[ ] Action costs - e.g. helicopter, how far last action was
[x] Reward Engineering
[x] Variable Radius Optimization - added radius 2 actions
[x] RGB image as input to the model with CNN - **Aidan finished this and will start training

williamshen-nz commented 2 years ago

Will:

[x] 2x2 actions
[x] Look into action masking for training. Hopefully this makes 40x40 work.
[x] Sweep over steps before simulation and steps per action for naive policy. Get some info on which params we should use for training
- Could use number of cells on fire fixed before applying any action. Done for Sub20x20 and Sub40x40

Aidan:

[x] Split bigger environments into more smaller ones, e.g. Dogrib to bunch of 60x60
- Will need to determine 'interesting parts' of the large maps that have diverse vegetation
[x] More complex plotting of tensorboard results
[x] MNIST - wow very cool!

Notes:

We will train on PPO, TRPO and A2C. PPO is slow
We're not learning interesting behaviors (though we are containing fire i.e. making a box which is interesting)
Evaluate policy duplicate actions. We should use same methodology for training/evaluating.
Action costs, complex rewards? Let's worry about it a bit later

williamshen-nz commented 2 years ago

Aidan

Will:

[x] Test action space box
[x] Masking actually helps?
[x] Debug process dying
[x] Frontier Baseline
[x] 20x20 stuff Decaying reward, playing with reward function. Autoregressive model

williamshen-nz commented 2 years ago

Imminent work for the slides on Thursday.

Aidan

Will

Then we discuss what's needed for the paper

aidan-curtis / firehose