search

MeMory-of-MOtion / docker-loco3d

Discussions relative to the docker for the generation of motion data using Loco3D pipeline.
1 stars 1 forks source link

Success Rates in the dataset of talos-stairs and talos-nav #3

Closed teguhSL closed 5 years ago

teguhSL commented 5 years ago

For the other two datasets, talos stairs and talos nav, I noticed that most of the examples fail to reach the goal. (I visualized the data set at random, but after many examples it is still the same)

This is especially true in the case of talos-stairs. Most of the examples fail after 1 or 2 steps on the stairs.

For talos-nav, the successful examples are the easy ones, i.e. moving in a straight line and in short distance. As soon as the task require the robot to turn to the right, the planner actually fails.

It seems that the contact generation also has problem in the case of talos-nav, because when the goal is specified to be on the other side of the stairs, the robot seems to head towards the goal directly, instead of turning around the stairs to find the way.

pFernbach commented 5 years ago

For the results of this version of the dataset, we expected a low success rate. That's why we recorded all the motions, even the ones not reaching the goal. However, some of the cases you describe seems rather easy to solve and should not have caused any issues.

Could you attach some infos.log files for some cases where you think that the problem was easily solvable and the planner fail ? I'll test them with our new framework once it's ready.

teguhSL commented 5 years ago

Ok, here are some examples of failed cases:

Case 1: long distance, the robot has to turn. 'd5d9c292-69a3-479c-bb7a-66f7deec1829', infos1.log.

infos1.log

Case 2: short distance, the robot has to turn. 'cb3d17df-a206-4e9a-8a37-5b6d251b4ac4', infos2.log.

infos2.log

Case 3: short distance, straight motion. 'dbe8d2fd-1324-449d-9913-10c3982b68e3', infos3.log.

infos3.log

Case 4: the target is behind the stairs, but the robot is heading towards the stair instead of turning around.

infos4.log

nmansard commented 5 years ago

For later dataset: I believe we should have ID numbers (hash) so that it is super easy to report an error and discuss a scenario without copying the data again. Could we implement that?