search

rr-learning / CausalWorld

CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning
https://sites.google.com/view/causal-world/home
MIT License
205 stars 25 forks source link

Trained baseline models perform poorly #100

Open zrjnz opened 2 years ago

zrjnz commented 2 years ago

Hello, I have trained the baseline models for all of the tasks and the results are only good for the pushing and picking tasks, and not even that good for the picking one. As for pick and place and stacking, the trained baseline model fails consistently. Is this expected behavior?

I ran the reproduce_experiments.py script and then evaluated each trained model with the evaluation pipeline. I can post the videos and the evaluation script if needed.

ftraeuble commented 4 months ago

Hi zrjnz,

Thanks for trying out the package and sorry for the late reply.

You should be able to reproduce the baseline results reported in the paper as shown here https://github.com/rr-learning/CausalWorld/tree/master/scripts We trained the baseline policies on curriculum 0 and 1 to verify this and as expected we were able to get the same results with slight variance due to the random seeds. After training the policies, the policies should behave similar to the ones here https://github.com/rr-learning/CausalWorld/tree/master/causal_world/actors.

Can you post the evaluation results using the evaluation protocols as well as the videos, so we can help you better.