search

microsoft / IBAC-SNI

Code to reproduce the NeurIPS 2019 paper "Generalization in Reinforcement Learning with Selective Noise Injection and Information Bottleneck" by Maximilian Igl, Kamil Ciosek, Yingzhen Li, Sebastian Tschiatschek, Cheng Zhang, Sam Devlin and Katja Hofmann.
https://arxiv.org/abs/1910.12911
Other
52 stars 17 forks source link

How to use Bottleneck layer? #1

Closed kayuksel closed 4 years ago

kayuksel commented 4 years ago

Hello, thanks for sharing this impressive work. I would be more than glad if you can explain how to use the proposed Bottleneck layer, as it is a bit difficult to understand that from the current code / paper e.g. where the Bottleneck layer should be placed in a Deep Learning architecture? It would be great to include a small toy example that demonstrates the usage. Sincerely, Kamer

maximilianigl commented 4 years ago

In our experiments, the bottleneck layer is placed second to last in the architecture, just before the value and policy heads of the network. For the layer itself, we follow the implementation proposed by A. Alemi, e.g. see here: https://github.com/alexalemi/vib_demo.

In our code, the implementation of the bottleneck can be seen well in the Multiroom environment, e.g. see here for the implementation of the layer and here for how it's used. We additionally return the mean (not only the sampled value), as we need it for the SNI implementation.

Hope that helps. Let me know if you have any further questions.