search

miyosuda / unreal

Reinforcement learning with unsupervised auxiliary tasks
Other
416 stars 131 forks source link

Feature Control #28

Closed kanishkg closed 5 years ago

kanishkg commented 5 years ago

The paper lists feature control as an auxiliary task. Quoting the paper,

We train agents that learn a separate policy for maximally activating each of the units in a specific hidden layer. We refer to these tasks as feature control.

The paper also does not provide any supplementary details as mentioned in the appendix to show how these can be implemented.

Has this auxiliary task been implemented? I cannot find it in the code.

Also Mnih mentions on OpenReview that

We have updated the paper with improved results for feature control. We used a target network to make the features being controlled change less frequently during training. Feature control now works roughly as well as pixel control.

kanishkg commented 5 years ago

Oh ok found on OpenReview Mnih mentions that:

UNREAL currently uses Pixel Control and no Feature Control. We decided not to put it into the main agent at this point since it is an ongoing research direction, with more preliminary results (as shown on Figure 5).

Leaving issue for other people who are confused.