search

rhett-chen / graspness_implementation

My implementation of Graspnet Graspness.
Other
129 stars 29 forks source link

Is a Higher or Lower Grasp Score Better? #23

Closed pmj110119 closed 5 months ago

pmj110119 commented 7 months ago

Dear author, thank you for your dedication and hard work on this open-source project!

I am reaching out to seek some clarification regarding the grasp score used in your code. I've observed that a grasp score of 0.1 indicates the highest quality, with the quality gradually decreasing from 0.1 to 0.9. Additionally, a score of 0 seems to represent a failure.

This scoring system seems a bit counterintuitive, as the worst score is 0, the best score is 0.1, and any increase from this point indicates a decrease in quality. This is not a straightforward increasing scale. In contrast, in the graspnetAPI, a higher score is typically considered better, which seems to create some ambiguity.

Indeed, I have used your code and trained a realsense model for 10 epochs. However, I only achieved 50 AP on seen objects. This result led me to wonder if there might be an issue related to the GraspnetAPI updates that could be causing a conflict with your current code.

Could you please confirm if this scoring arrangement is intended? I am concerned that I might be misunderstanding something and would greatly appreciate your patient explanation.

Best regards.

rhett-chen commented 7 months ago

The score annotated in the GraspNet dataset refers to the friction coefficient at which an object can be grasped, and the lower the better. We have already converted it to higher is better in the code, you can refer to code.

pmj110119 commented 7 months ago

I understand, thank you for the explanation!

However, I can't achieve an comparable AP score mentioned in the paper (Realsense, AP=65), and I'm uncertain what's causing this. This repository only reports the AP score for Kinect. Do you have the AP for Realsense?

My experimental results: Batch size = 4, trained for 10 epochs AP (seen, 100-130) = 55

rhett-chen commented 7 months ago

Hi, you can check if the generated grassness is correct? It can be checked through point cloud visualization.

pmj110119 commented 7 months ago

It seems right: image

(The higher the graspness, the whiter the color of this point.)

pmj110119 commented 7 months ago

I found the trained checkpoints in another repo (forked from your repo), but it only has 20 AP score for evaluation.

Is there a mismatch between the training code and the current version code?

rhett-chen commented 5 months ago

I'm not sure. The training checkpoints you are referring to were not provided by me.