search

Emerge-Lab / gpudrive

GPU-acceleration of Nocturne via Madrona
https://arxiv.org/abs/2408.01584
MIT License
211 stars 19 forks source link

Suggested Controller #268

Open JohnZhan2023 opened 1 week ago

JohnZhan2023 commented 1 week ago

If the output of my model is a trajectory, how soon will the repo support converting the trajectory to the discrete action? If it is not soon, what strategy will you suggest me to bridge the gap between trajectories and actions?

JohnZhan2023 commented 1 week ago

additionally, I notice that on one 3090, the num of world can only be 100, which means I can only train my model on 300 scenes out of 130000 scenes. How can I iterate train the whole set?

eugenevinitsky commented 1 week ago

Hi John, there is not one obvious and unique way to convert between a trajectory and discrete actions. We already support inverse kinematics so for a particular change in x-y position we can invert and compute the actions that will give you that. Perhaps you could be more specific about what exactly you're looking for? As for the latter question, a PR is coming https://github.com/Emerge-Lab/gpudrive/pull/265

JohnZhan2023 commented 6 days ago

Is the dynamic world file reseting implemented automatically in the sb3 ppo algorithm provided? Or if I want to use it, I need to add it to my training pipeline manually?

By the way, I wonder why you choose the discrete action space instead of continuous ones?

JohnZhan2023 commented 6 days ago

Besides, in jupyter notebook 5, the expert policy use the continuous action, but I notice the continuous action is not supported. I am confused.

eugenevinitsky commented 4 days ago

Dynamic world resetting is not supported yet. It will be after that PR is completed. We chose a discrete action space to default to as historically discrete RL is easier than continuous RL. However, continuous actions are supported!

daphnecor commented 4 days ago

Besides, in jupyter notebook 5, the expert policy use the continuous action, but I notice the continuous action is not supported. I am confused.

I see the confusion. As @eugenevinitsky mentioned, continuous actions are now supported. I will update the docs

JohnZhan2023 commented 4 days ago

Hi John, there is not one obvious and unique way to convert between a trajectory and discrete actions. We already support inverse kinematics so for a particular change in x-y position we can invert and compute the actions that will give you that. Perhaps you could be more specific about what exactly you're looking for? As for the latter question, a PR is coming #265

so， where is your implementation of inverse kinematics and how soon will the Dynamic world resetting be available?

eugenevinitsky commented 3 days ago

A little unclear, it's a tricky issue and we want to make sure it lands without bugs. In terms of inverse kinematics, we have a bunch of them here: https://github.com/Emerge-Lab/gpudrive/blob/main/src/dynamics.hpp

daphnecor commented 1 day ago

Hi @JohnZhan2023, we have just merged the feature that enables dynamic setting traffic scenes in the simulator! This feature will allow you to train/iterate through the full dataset.

Please find the documentation at pygpudrive/env/README.