Screwing task: success criteria for "aligning screw into nut" and "tightening screw"

[abhishek47kashyap]

For the last two sub-tasks of the screwing task (Align screw into nut and Tighten the screw), can more information be provided on how is alignment evaluated, and how is it determined whether screw is tightened or not?

Picture below shows the screw placed on the nut in a way that to me looks "aligned". (It's not at all tightened yet.)

However, Align screw into nut is not considered complete. According to /get_assembly_status:

$ rosservice call /get_assembly_status                 
status: 0
stage_times: [5.651000000000222, 3.3900000000002066, 6.085999999996627, -1.0, -1.0]

Here are two _GT positions, if relevant:

$ rostopic echo -n1 /nut_head_pos_GT/point
x: 0.5476486086845398
y: -0.04500710219144821
z: 0.38099977374076843
---
$ rostopic echo -n1 /screw_pos_GT/point
x: 0.5481606125831604
y: -0.045070141553878784
z: 0.38730737566947937
---

I'm unsure what more would I have to do to complete Align screw into nut. And for when I get to Tighten the screw, I'm also unsure how to know whether the screw is indeed getting tightened.

[DavidPL1]

Our member in charge of the screwing task scoring and threading simulation is currently out of office, but I'll try my best to give a good answer:

I've just tested the setup and it seems that alignment means near perfect alignment of x and y axes within a larger z-axis offset region. Your y GT position seems good, but probably the 0.05 cm offset on the x axis are already too big. But your image looks okay to me, so we might have to increase the allowed margin in the alignment. We'll have to discuss that internally.

However, once you start tightening the screw, alignment should at the latest be considered complete once the simulated threads align rotation-wise (the objects are then welded together at the correct relative coordinates) and threading begins.

Hope this helps!

[abhishek47kashyap]

Thanks for the info.

but probably the 0.05 cm offset on the x axis are already too big

I've added alignment corrections to keep misalignment in X and Y under a threshold, it'll be good to know what threshold is being used internally to decide whether Align screw into nut is complete or not.

(On a more general note, it might be helpful to provide some "feedback" through /get_assembly_status or some other way on what remains for a subtask to be considered complete.)

[Director-of-G]

We have experienced the same issues. The evaluation criteria of both tasks is ambiguous.

In the screwing task, for example, we 'aligned' and 'tightened' the screw and nut based on our understanding, as shown in the figure below.

Note that the black sealing ring on the screw fit smoothly between the screw and nut, however, the time consumption for 'Align screw into nut' and 'Tighten the screw' subtasks are still -1.0, which indicates they are not completed.

Similar concerns apply to the plugging task, as simple discriptions do not provide enough information about what we should do to complete these subtasks, like 'Localize the frame', 'Click-insert the plug' and so on.

I wonder if you could provide more detailed explanation of subtask completion criteria (like figures of completed cases, and possible misunderstandings). These information are very important to us. Thank you so much.

[GostInShell]

Hi! I experience a similar problem but not exactly the same as @abhishek47kashyap and @Director-of-G .

For the screw task, I find that the evaluation of all stages run smoothly in docker.assembly_server-v1.1.tar.gz, when we successfully complete the task. However, in the docker V2.1, our code fails to complete the tightening stage, even when it uses the GT pose of the object. It seems that the screw just would not go into the nut even if we keep rotating the screw along the z axis.

Besides, for the evaluation of aligning the screw with the nut stage, we find that, in docker v1.1, the terminal does not print the task success message until the screw spins around once during tightening. But in docker v2.1, the aligning the screw with the nut stage is never completed even if we keep rotating the screw.

Btw, I wonder which version of docker does @Director-of-G use, since you can successfully complete the tighteningstage? Thanks.

Hope these information helps to resolve the problem.

Thanks.

[DavidPL1]

@abhishek47kashyap

I've added alignment corrections to keep misalignment in X and Y under a threshold, it'll be good to know what threshold is being used internally to decide whether Align screw into nut is complete or not.

I looked through the code and found that these two criteria have to be met:

1. the cartesian distance between nut head and screw tip must be below 0.002m
2. the roll and pitch offset between screw and nut both have to be below $\frac{\pi}{9}$ degrees
3. then the screw might need to be turned until its simulated thread is correctly aligned (rotation-wise of course) with the nut

For this last criterion there is no way for you to measure this. But in case your initial alignment is not correct, rotating the screw in place is trivial. This then locks the screw and nut together and starts simulating the threading process upon changing their relative yaw angle.

(On a more general note, it might be helpful to provide some "feedback" through /get_assembly_status or some other way on what remains for a subtask to be considered complete.)

I don't think we are going to add that, because the risk of the feedback being exploited to finish the sub-tasks is too high. We'd rather have participants design reward functions which take into account the (noisy) sensor information that is available. Realizing when a sub-task is done and moving on to the next one can also be seen as a sub-challenge to optimize timing.

---

@Director-of-G & @GostInShell Indeed there was an error in one of the build scripts which caused the wrong branch to be built. I've uploaded version 2.2-pre, with which I got the correct the correct feedback for the aligning and tightening stage.

The attached videos displays an example of the full task and includes the get_assembly_status feedback output:

https://user-images.githubusercontent.com/20286447/230891750-2b2f6d94-a336-4fa2-959a-3b0eaa5f1ecf.mp4

Edit: If your Browser tells you the video is corrupt, try another browser. For me it does not work in FF, but in Chromium

[abhishek47kashyap]

Alright, thanks for the info and the video.