Continuous time supported?

[JinraeKim]

I'm looking forward some RL packages incorporating continuous-time dynamics (or, environments). I'm too new to this package so this functionality may be already realised.

For example, we may implement RL algorithms to control quadcopters with zero-order-holding (ZOH)-like implementation. In this case, some hyperparameters, such as time duration between each execution of actions, should be provided. Is it possible with this package?

Note that this does not mean that our RL algorithms should be formulated as continuous-time algorithm as there are many methodology for discrete-time policy to continuous-time environments control logics, e.g., ZOH control.

Thanks.

+) If the functionality has not been realised and you (developers) take into account it, I highly recommend you to make it compatible with DifferentialEquations.jl for Julia ecosystem.

[findmyway]

Hi @JinraeKim ,

For the algorithms part, PPO and DDPG are for continuous control related tasks. I'm no expert in automatic control, but I think they will at least work as a baseline.

For the environment part, this package only provides a set of standard interfaces (RLBase) and some typical implementations (RLEnvs). The only continuous-time environment we have is a simple pendulum environment. To use the algorithms provided in this package, you just need to provide a wrapper to the underlying system (it can be implemented with whatever dependencies you want or even in other languages).

It's not very clear to me what features you are looking for. But I'd be glad to provide help.

[JinraeKim]

Thanks a lot! I would have to try this repo :) As your suggestion, I'll try what I want on the simple pendulum env.

I'll close this issue as I have to understand this package for now to determine the functionality is provided.