Making Tasks Infinite Horizon

nileshop22 commented 2 years ago

Hi, from the White Paper I found that these tasks terminate after 1000 steps. I'm working with Infinite Horizon Problems for which I need continuing/ infinite-horizon/ never ending tasks. Is there any way by which I can remove the 1000 steps constraint.

Also, I want to train a DQN agent for which I need discrete actions. Is there any way of discretezing the continuous action space for this suite?

Any help would be really appreciated. Thanks.

ethanluoyc commented 2 years ago

Regarding the first question, I believe that you there is an option to specify a time limit, see for example https://github.com/deepmind/dm_control/blob/4f1a9944bf74066b1ffe982632f20e6c687d45f1/dm_control/suite/cartpole.py#L62

To use this, I think something like

from dm_control import suite

suite.load(
  "pendulum", 
  "swingup", 
  task_kwargs={"time_limit", float("inf"))},
)

would work.

yuvaltassa commented 2 years ago

Indeed, you can take a look at the LQR task, which has a terminal state condition rather than a time limit.

On Thu, 11 Nov 2021 at 23:13, Yicheng Luo @.***> wrote:

Regarding the first question, I believe that you there is an option to specify a time limit, see for example

https://github.com/deepmind/dm_control/blob/4f1a9944bf74066b1ffe982632f20e6c687d45f1/dm_control/suite/cartpole.py#L62

To use this, I think something like

from dm_control import suite suite.load( "pendulum", "swingup", task_kwargs={"time_limit", float("inf"))}, )

would work.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/deepmind/dm_control/issues/213#issuecomment-966690993, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABQXORUCFLMYBTH4UMAYFVDULRE3FANCNFSM5HLDIQEQ .

--

Dr. Yuval Tassa | Research Scientist | Google DeepMind | @.***

google-deepmind / dm_control

Making Tasks Infinite Horizon #213