search

microsoft / FQF

FQF(Fully parameterized Quantile Function for distributional reinforcement learning) is a general reinforcement learning framework for Atari games, which can learn to play Atari games automatically by predicting return distribution in the form of a fully parameterized quantile function.
Other
40 stars 10 forks source link

Reproducing paper results #1

Open nino-vieillard opened 4 years ago

nino-vieillard commented 4 years ago

Hi, I am trying to evaluate FQF, to use it as a baseline on some discrete environments. However, I encountered an issue: the script run-iqn.sh [EDIT: run-fqf.sh] does not seem to evaluate FQF, but actually IQN. I think the problem comes from the function create_agent in dopamine/discrete_domains/run_experiment.py can only create Rainbow, DQN and IQN (and not FQF). It is possible I missed something, could you explain how I can use this code to evaluate FQF? Thanks, Nino

waterblue13 commented 4 years ago

Hi, I am trying to evaluate FQF, to use it as a baseline on some discrete environments. However, I encountered an issue: the script run-iqn.sh does not seem to evaluate FQF, but actually IQN. I think the problem comes from the function create_agent in dopamine/discrete_domains/run_experiment.py can only create Rainbow, DQN and IQN (and not FQF). It is possible I missed something, could you explain how I can use this code to evaluate FQF? Thanks, Nino

Maybe you can try run-fqf.sh.

nino-vieillard commented 4 years ago

Hi, I am trying to evaluate FQF, to use it as a baseline on some discrete environments. However, I encountered an issue: the script run-iqn.sh does not seem to evaluate FQF, but actually IQN. I think the problem comes from the function create_agent in dopamine/discrete_domains/run_experiment.py can only create Rainbow, DQN and IQN (and not FQF). It is possible I missed something, could you explain how I can use this code to evaluate FQF? Thanks, Nino

Maybe you can try run-fqf.sh.

Hi, Sorry for the typo, I meant that run-fqf.sh seems to run IQN, not FQF.

LinZichuan commented 4 years ago

Hi, I am trying to evaluate FQF, to use it as a baseline on some discrete environments. However, I encountered an issue: the script run-iqn.sh does not seem to evaluate FQF, but actually IQN. I think the problem comes from the function create_agent in dopamine/discrete_domains/run_experiment.py can only create Rainbow, DQN and IQN (and not FQF). It is possible I missed something, could you explain how I can use this code to evaluate FQF? Thanks, Nino

Maybe you can try run-fqf.sh.

Hi, Sorry for the typo, I meant that run-fqf.sh seems to run IQN, not FQF.

Thanks for pointing out this issue!

This is caused by a typo (agent_name='implicit_quantile' should be agent_name='fqf') in agents/fqf/configs/fqf.gin when we refactor our code. And also you can add an FQFAgent in the create_agent function of dopamine/discrete_domains/run_experiment.py.

We will update the code later. Thanks!

LinZichuan commented 4 years ago

The code has been updated. @nino-vieillard