Issues with running notebooks/boxing_example.ipynb

[susmitjha]

There are a few issues with running notebooks/boxing_example.ipynb

1. trojai_rl has an older gym version (0.15.3) requirement, but it appears one needs gym[accept-rom-license]==0.21.0 to get the Atari ROMs. Consequently, the numpy requirement also needs to be updated.
2. trojai_rl.datagen.envs.wrapped_boxing_public should be trojai_rl.datagen.envs.wrapped_boxing and trojai_rl.modelgen.architectures.atari_architectures_public is perhaps trojai_rl.modelgen.architectures.atari_architectures
3. Running train_boxing_model() produces the following error: TypeError: only integer scalar arrays can be converted to a scalar index The problem seems to be in torch_ac_optimizer.py. The code below produces a torch array when using argmax if argmax_action: action = torch.argmax(dist.probs) # get 'best' decision else: action = dist.sample() Using action = action.numpy()[0] before passing it to the step function appears to fix the issue, since the step function is expecting a scalar.

With these changes, we get qualitatively similar results on the new model trained by the notebook and the pretrained ones.

[GongYanfu]

How long did you get the result? please