Suggested Controller

[JohnZhan2023]

If the output of my model is a trajectory, how soon will the repo support converting the trajectory to the discrete action? If it is not soon, what strategy will you suggest me to bridge the gap between trajectories and actions?

[JohnZhan2023]

additionally, I notice that on one 3090, the num of world can only be 100, which means I can only train my model on 300 scenes out of 130000 scenes. How can I iterate train the whole set?

[eugenevinitsky]

Hi John, there is not one obvious and unique way to convert between a trajectory and discrete actions. We already support inverse kinematics so for a particular change in x-y position we can invert and compute the actions that will give you that. Perhaps you could be more specific about what exactly you're looking for? As for the latter question, a PR is coming https://github.com/Emerge-Lab/gpudrive/pull/265

[JohnZhan2023]

Is the dynamic world file reseting implemented automatically in the sb3 ppo algorithm provided? Or if I want to use it, I need to add it to my training pipeline manually?

By the way, I wonder why you choose the discrete action space instead of continuous ones?

[JohnZhan2023]

Besides, in jupyter notebook 5, the expert policy use the continuous action, but I notice the continuous action is not supported. I am confused.

[eugenevinitsky]

Dynamic world resetting is not supported yet. It will be after that PR is completed. We chose a discrete action space to default to as historically discrete RL is easier than continuous RL. However, continuous actions are supported!

[daphnecor]

Besides, in jupyter notebook 5, the expert policy use the continuous action, but I notice the continuous action is not supported. I am confused.

I see the confusion. As @eugenevinitsky mentioned, continuous actions are now supported. I will update the docs

[JohnZhan2023]

Hi John, there is not one obvious and unique way to convert between a trajectory and discrete actions. We already support inverse kinematics so for a particular change in x-y position we can invert and compute the actions that will give you that. Perhaps you could be more specific about what exactly you're looking for? As for the latter question, a PR is coming #265

so， where is your implementation of inverse kinematics and how soon will the Dynamic world resetting be available?

[eugenevinitsky]

A little unclear, it's a tricky issue and we want to make sure it lands without bugs. In terms of inverse kinematics, we have a bunch of them here: https://github.com/Emerge-Lab/gpudrive/blob/main/src/dynamics.hpp

[daphnecor]

Hi @JohnZhan2023, we have just merged the feature that enables dynamic setting traffic scenes in the simulator! This feature will allow you to train/iterate through the full dataset.

Please find the documentation at pygpudrive/env/README.

[eugenevinitsky]

Closing this issue if there aren't updates here? Feel free to reopen.