search

YanjieZe / 3D-Diffusion-Policy

[RSS 2024] 3D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representations
https://3d-diffusion-policy.github.io
MIT License
374 stars 33 forks source link

Reproduce results #22

Closed zhouzheyuan closed 4 months ago

zhouzheyuan commented 4 months ago

Hi Yanjie, thanks for your great work!

Currently, I'm reproducing the results of DexArt and I'm noticing that the results seem to be heavily influenced by the generated demonstration. IMG_0030 Specifically, Exp. 0322 and Exp. 0430 used the same demonstration, where Exp. 0430 was repeated five times using different seeds. Exp. 0501 regenerated the demonstration.

It can be seen that the experimental results for the different demonstrations are varied and both are lower than the results reported in the paper. I wonder if there is a way to stabilized reproduction results.

YanjieZe commented 4 months ago

Hi, how many demonstrations do you use? In DexArt we actually use 100 demonstrations (see Table III).

I think the instability is led by the small number of demonstrations. We would consider releasing a few demos so to help better stabilize results.

zhouzheyuan commented 4 months ago

Hi, thanks for your quick response. I'll try as you suggest.

I followed the script you provided for DexArt, it seems like generating 10 demonstrations in total. https://github.com/YanjieZe/3D-Diffusion-Policy/blob/c77b6fb61f077dfcaef556da7df2c9653f24c537/scripts/gen_demonstration_dexart.sh#L10

YanjieZe commented 4 months ago

Hi, thank you for reminding:) I have changed that