search

nickgkan / 3d_diffuser_actor

Code for the paper "3D Diffuser Actor: Policy Diffusion with 3D Scene Representations"
https://3d-diffuser-actor.github.io/
MIT License
159 stars 16 forks source link

Relative_action #25

Closed chenjiayi-zxx closed 2 months ago

chenjiayi-zxx commented 2 months ago

Hello, I am currently exploring your fascinating project and I have noticed that the relative_action parameter is specifically utilized with CALVIN. I am curious to understand the rationale behind this specific usage.  Could you kindly share the insights or analyses that led to the decision to use relative_action only in the context of CALVIN?  Thank you very much for your time and for the impressive work on this project.

nickgkan commented 2 months ago

Hi @chenjiayi-zxx, thanks for you interest in our project!

In some earlier experiments we found using a relative coordinate frame not to be an important factor on RLBench. However, we did find it important on CALVIN, as also suggested by related literature such as HULC. We hypothesize that this is due to the nature of instructions of CALVIN, e.g. "mover block to the right", which suggest relative actions. Instead, on RLBench there is usually a set of solutions defined visually, e.g. reaching the handle of a door.

We haven't retried relative actions recently on RLBench, so it's possible that if you try you'll notice some performance difference.

chenjiayi-zxx commented 2 months ago

@nickgkan I understand, thank you very much for your reply.