Open SWAGamaz opened 11 months ago
Hello, Could you share what/how you mounted the markers onto your medical device?
Thanks Saharsh
Did you calibrate the cameras? That could be a reason why the tracking isn't working properly.
Greetings, @ssbarve2!
For mounting the markers, I used a ready-made 3D model of the tool with designated spots for reflective spheres, which I printed with a 3D printer. In these allocated spots, I installed infrared LEDs with diffusers. I am wondering whether the order of x, y, z coordinates for each point affects the system's ability to accurately identify the tool.
As of now, I haven’t calibrated the cameras and am trying to achieve at least some recognition of the tool, even if the accuracy isn’t high.
May I ask if you have succeeded in tracking your tool effectively? Is there anything else you could recommend or clarify to help me get a more complete picture of the situation?
Thank you in advance for your assistance!
Hello @SWAGamaz, I am currently in the initial stages of setting up the tool tracker myself, will keep you updated about my progress. We can connect via mail (if that's okay with you) to rectify each others doubts.
Currently, I am working on creating a basic instrument to track. Do you mind sharing your 3d model so that we can alter cross check our results.
Regards, Saharsh
Also, Any reason why you have placed markers on the same plane?
Regards, Saharsh
Hello @SWAGamaz,
there could be a couple of reasons as to why the tool is not detected. Since your camera images are a bit different from mine, maybe the parameters in the BlobFinder need to be adapted. As @ssbarve2 pointed out, the camera calibration might also be an issue. You also mentioned that you input the points in the Unity coordinate system. However, the OpenCV coordinate system is used within the Plugin, so you might have to invert the y-axis of your points!
For debugging it can be helpful to write a testing script that allows you to process pre-recorded frames offline. This makes it easier to visualize the intermediate steps.
Hello, I had a couple of doubts:
// definition of the model points
cv::Mat pointsModel = (cv::Mat_<float>(5, 3) <<
2.57336e-7f, 2.42391e-6f, 3.36463e-6f,
0.0442368f, 0.0349529f, 0.0726091f,
-0.0936104f, 0.0113667f, 0.0732692f,
0.000176568f, -0.000130322f, 0.108399f,
-0.00748328f, 0.0856192f, 0.0642646f);
This is how I am considering the instrument coordinate system.
Thanks
Good day! I need your help and advice. I sort of successfully built a project on OpenCV 4.8.0 for ARM64 and I'm trying to use it in Unity, but my tool is not tracked, the resulting ToolPosition array returns 0000000.
I was unable to attach infrared LEDs to the glasses in such a way as to get a good reflection from the reflective sphere. I took the opposite approach and mounted LEDs with a diffuser on the tool. From what I can tell by the image, they are very visible.
It seems to me, perhaps, the problem is in the incorrect identification of the tool's points. I took the coordinates from the model in the Unity coordinate system. Should I use a different coordinate system? And how should I input the coordinates so that, for instance, ToolPosition would return the position of the tool tip?
I really hope for your advice!
P.S. I'm providing a link to the compiled opencv 4.8.0 for ARM64. I built only those modules that are required by the project's dependencies. During the compilation, following your instructions, I didn't encounter any errors. However, I'm not sure how to verify that everything is working correctly.