search

facebookresearch / ReAgent

A platform for Reasoning systems (Reinforcement Learning, Contextual Bandits, etc.)
https://reagent.ai
BSD 3-Clause "New" or "Revised" License
3.56k stars 516 forks source link

Could you please provide some example on using "Data Understanding Tool"? #130

Open ruiwang2uber opened 5 years ago

ruiwang2uber commented 5 years ago

Thanks for updating the paper.

In the updated paper, you mentioned you have implemented a "data understanding tool" based on world model. This is super useful and important.

Could you provide some example on how to use the tool?

czxttkl commented 5 years ago

Sure. We will have a related diff coming out soon. Once it is done, you'll be able to run ml/rl/test/gym/world_model/mdnrnn_gym.py to get feature importance/sensitivity of any gym environment.

czxttkl commented 5 years ago

Expect timeline: 2 weeks.

ruiwang2uber commented 5 years ago

Thank you! Look forward to it!

ruiwang2uber commented 5 years ago

Any updates on "Data Understanding Tool"? Thanks.

czxttkl commented 5 years ago

Hi,

Sorry for some delay. Now you can use the following command to check feature importance and feature sensitivity in the cartpole environment: python mdnrnn_gym.py -- -p ml/rl/test/configs/mdnrnn_cartpole_v0.json -f -s

czxttkl commented 5 years ago

The output should be something like this:

INFO:ml.rl.evaluation.world_model_evaluator: Debug tool feature importance : tensor([18.9650, 9.1601, 0.1277, 13.4917, 0.7331, 12.0240]) [2019-06-19 13:48:59,942] Debug tool feature importance : tensor([18.9650, 9.1601, 0.1277, 13.4917, 0.7331, 12.0240]) action 0, feature importance: 18.964975357055664 action 1, feature importance: 9.16006088256836 state 0, feature importance: 0.1277211308479309 state 1, feature importance: 13.491726875305176 state 2, feature importance: 0.7331321239471436 state 3, feature importance: 12.024038314819336 INFO:ml.rl.evaluation.world_model_evaluator: Debug tool feature sensitivity : tensor([0.0067, 0.2099, 0.0066, 0.3125]) [2019-06-19 13:49:07,774] Debug tool feature sensitivity : tensor([0.0067, 0.2099, 0.0066, 0.3125]) state 0, feature sensitivity: 0.006689509842544794 state 1, feature sensitivity: 0.2099362462759018 state 2, feature sensitivity: 0.00659541692584753 state 3, feature sensitivity: 0.31252968311309814