search

Lifelong-Robot-Learning / LIBERO

Benchmarking Knowledge Transfer in Lifelong Robot Learning
MIT License
163 stars 23 forks source link

Negative Loss, Zero Success and Zero AoC #21

Open bit0123 opened 1 month ago

bit0123 commented 1 month ago

Hi, Thanks for the interesting paper and releasing the code. I am facing the same issue ("TypeError: h5py objects cannot be pickled") while running with num_worker=4. However, the code works when i set num_worker=0 with some additional transformation of input image in the policy file.

The loss becomes negative after the first epoch for different algorithm with 0 success. Consequently, the AoC is 0. Could you please comment on that. Thanks

wangair commented 3 weeks ago

Hi, Thanks for the interesting paper and releasing the code. I am facing the same issue ("TypeError: h5py objects cannot be pickled") while running with num_worker=4. However, the code works when i set num_worker=0 with some additional transformation of input image in the policy file.

The loss becomes negative after the first epoch for different algorithm with 0 success. Consequently, the AoC is 0. Could you please comment on that. Thanks

Hi!Have you solved this?

BrightMoonStar commented 2 weeks ago

I encountered the same problem. All the evaluation indicators were 0, including AOC, Success Rate ...

BrightMoonStar commented 2 weeks ago

What confuses me is that the following two projects based on LIBERO also have success rate = 0 and Aoc = 0. No matter how we train, the result is 0. Is it because the details of our configuration are incorrect? I would like to ask for your help. Thank you very much! @Cranial-XIX https://github.com/UT-Austin-RPL/Lotus

pengzhi1998 commented 1 week ago

After 20 epochs of training, the success rate would become larger than 0. And after 50 epochs of training, the average success rate is around 0.7. (for libero_object tasks)

BrightMoonStar commented 1 week ago

经过 20 个时期的训练后，成功率将大于 0。经过 50 个时期的训练后，平均成功率约为 0.7。（对于libero_object任务）

Have you ever tried libero-90? For libero-90, it seems the problem still exists for me. Thank you very much!

Cranial-XIX commented 1 week ago

The success rate on libero-90 is low by design (since there are 90 tasks), and in general, you might need over 20 epochs (like 25 - 35) to reach the peak result.