Open sean1295 opened 1 month ago
The problem of 100% accurate action space conversion is highly non-trivial (if you solve it it's worth a conference publication). It is quite difficult to ensure the resulting environment state from using a different controller is the same as the original trajectory data, we can only do approximations and at best, try to get a success rate that matches the trajectory data
Hi Stone, thanks for your prompt reply. I agree. When trying to convert from pd_joint_pos to pd_ee_delta_pose, I noticed that ~1% of them didn't convert successfully, which I am totally fine with since I can simply generate more trajectories from original env. But this seemly easy conversion from pd_joint_delta_pos to pd_joint_pos actually has 0 % conversion rate. I think this particular conversion has a lot of utility since many RL agents can be trained with pd_joint_delta_pos and we may want to convert that to other action spaces. In order to do so, we first need to convert that back to pd_joint_pos, and then the target action space.
If it has 0 then that sounds like a bug. I'll keep the issue up and take a look soon.
In replay_trajectory.py, when trying to convert environment in pd_joint_delta_pos to pd_joint_pos using the following commands,
python -m mani_skill.trajectory.replay_trajectory --traj-path demo/StackCubeEnv/20240716_102711.h5 --save-traj -o rgb --target-control-mode pd_ee_delta_pose --num-procs 4 --use-first-env-state --max-retry 5
it keeps giving me following messages: "Episode xxx is not replayed successfully. Skipping."
I also kept track of the joint positions of the robots across two environments using the converted actions and they do not follow exactly. Please see below.![스크린샷 2024-07-16 163116](https://github.com/user-attachments/assets/f91c15db-e8c3-442d-b2f4-7890c0d317eb)
Was looking at the code but do not see anything that stands out. aside from one error inside clip_and_scale_action function where clip method from numpy should be used instead of torch.
Am wondering if there are ways to fix this.