Closed assia855 closed 1 month ago
Hi @assia855,
Some general tips:
Hi @cviviers Thanks for all this suggestsion.
You need to print the exact pdf/png that is produced by the script. Do not cut the markers out and put them in different places or on a new piece of paper. The distance between markers also matters. Your latest pictures also seem like the paper is not flat enough.
I think the offset you are referring to is the offset to the object from marker 1. Change this when you see the correct axis of the board in the corner of the board being displayed.
Hi @cviviers thanks a lot for your replay, I printed the charuco board and recompute the camera parameters with calibrate.ipynb this is the parameters that I got: {"distortion": [-0.04745589091727264, 0.0004570602824454296, 0.012062003741561756, 0.008462426231645262, -1.1409376245420838e-06], "intrinsic": [[133.8815378174932, 0.0, 326.066571194312], [0.0, 81.07665840500025, 239.59919600396822], [0.0, 0.0, 1.0]]}
and after running create_dataset.ipynb with your parameters this is the projection that I got:
What can I fix as well to have the exact pose please?
Thanks a lot for your help.
what aruco dictionary did you use? remember since you created a custom board, you should use that custom board in all the other scripts as well.
so for calibrate: instead of the current (# aruco_dict = Aruco.getPredefinedDictionary(Aruco.DICT_6X6_50) board = Aruco.CharucoBoard((8, 5), 0.5, 0.3, aruco_dict) ), use the board description you printed. the same holds for the create dataset.
I can already see you are only using Aruco.DICT_6X6_50, instead of the Aruco.DICT_6X6_250 used in the create_board script.
Hi @cviviers thanks for the remark, yes actually you are right I didn't pay attention to that but actually in your code calibrate for the board you have: board = Aruco.CharucoBoard((8, 5), 0.5, 0.3, aruco_dict)
and in pose_utils you have: def detect_Charuco_pose_board(img, matrix_coefficients, distortion_coefficients, markerSize = 6, totalMarkers = 50, number_markers_x = 8, number_markers_y = 5, square_width = 0.05, aruco_width = 0.03)
what's the reason for this mismatch between the two scripts for aruco_width and square_width parameters.
aruco_dict = Aruco.getPredefinedDictionary(Aruco.DICT_6X6_250)
board = Aruco.CharucoBoard((10,10), 0.03, 0.02, aruco_dict)
Then the new camera parameters are:
def detect_Charuco_pose_board(img, matrix_coefficients, distortion_coefficients, markerSize = 6, totalMarkers = 250,
number_markers_x = 10, number_markers_y = 10, square_width = 0.03, aruco_width = 0.02)
but I still have this kind of projection:
What I'm still doing wrong please? Thanks a lot :)
You do not have to use the same board to calibrate your camera and to eventually acquire data with. You can just use a standard checkerboard to calibrate the camera. In your implementation above, do you also use theboardpts, num
from when you created the custom board in the follow up code?
I think to ensure everything is well calibrated and to acquire the board pose correctly, lets start with the basic charuco board first (no cup included). From now on, use the board you describe here
aruco_dict = Aruco.getPredefinedDictionary(Aruco.DICT_6X6_250)
board = Aruco.CharucoBoard((10,10), 0.03, 0.02, aruco_dict)
in all your scripts and also reprint it. It will have no hole in the middle but thats okay.
This should give you the correct poses I believe.
I will try to update the getting started code in the coming weeks to streamline the process. I think for people that are new to aruco markers and pose estimation there are quire some pitfalls. Your feedback helps a lot :)
Hi @cviviers thanks for your answer, yes I use as well the boardpts, num
to generate the charuco board.
I follow your suggestion:
calibrate.ipynb
to generate camera_parameters.json
file rotation and translation offset
than I continue the process for the projection I got :
but for the mask I got:
Do the projection results indicate that my labeling are correct now? Could you please advise me on how to fix the mask issue?
Thanks for all your help.
Very good, nice job getting the pose of the board. So the next part becomes a bit more tricky, the current code gives you this box and mask with which you can already train. However, it will likely not work well when deployed in a real world scenario. Do you have an accurate 3D model of the mug? If so, there are a few ways you can project that model onto the image to get an accurate mask
Hi @cviviers thanks a lot. Actually I launch train with this mask it trains and I have a detection on test image but I don't understand why I have can't open/read file: check file path/integrity
in terminal while training. Am I missing somthing? should I register the images somewhere first before training?
For the 3D model of the mug I got it from YCB-V dataset. I'm trying to find some codes to project the 3D on my 2D images but I'm not sure it is the good way to that or I should generate a 3D model of my real mug. I took the 3D mug of YCB-V dataset because it was similar to my mug, clean and respect all the details that the neural needs.
thanks so much for your help and illumination.
This is a discrepancy between the windows and linux pathing convention I think. Somewhere either a \
is being used when it should be a /
Delete the cache files and check which file is not using the linux file pathing convention.
thanks @cviviers,
I solve this issue effectively it was just issue of /
when I switched it the warning disappear.
I recently thought of a nice method for you to get accurate enough masks if you do not have a good 3D model. You can just use the segment anything model https://github.com/facebookresearch/segment-anything to set up a semi automatic segmentation of your objects. You can just prompt is with "cup" in each image and then check if it gives you good results.
Thanks @cviviers, actually for the mask I used track anything and I generate the masks then I launch a training with it, here's the results.
but I don't know if the rotation and the translation of the object is correct or not since I don't visualize as well the mask on the object?
If you run the test.py instead of detect.py it will draw the ground truth points as well if you have them
Hi @cviviers I run test.py and it create an empty folder in test I don't understand why ?
Thanks a lot for your help!
You have to pass the flag --test-plotting
when running test.py. Please check the code
thanks @cviviers for your remark. Actually now I have the images with two boxes blue and green but no mesh. Why I don't have the projection of the mesh ?
The current code does not do that, since we were interested in obtaining the pose using the keypoints. Feel free to extend the code to your needs :)
With this I think you've learnt a lot and completed what your current Github issue set out to do. Congrats! Feel free to open a new issue if you have any more problems with the current code.
Hi @cviviers, could you please share the Charuco board you use in create_dataset I'm struggling with mine. Even after changing the detect_Charuco_pose_board parameters from number_markers_x = 8, number_markers_y = 5 to number_markers_x = 4, number_markers_y = 3, it only computes the pose for a few frames in my dataset. Additionally, when I test the poses of the frames I obtained in Sanity check function, I get these kinds of poses.