Azure Kinect calibration problem

[Zetee]

Thanks for sharing this great work! I have compiled the Azure Kinects brunch and testing for two Azure Kinects. In calibration step, the clients can detect the same mark id 0 (green border around the marker in both clients), but after added mark id 0 in setting, calibrate never worked, as the green border didn't disappear and no calibration file generated. Do I need to add the orientation and translation info? I am really appreciated if you can give some suggestions.

[MarekKowalski]

Hi,

Did you compile in Debug or Release? When compiling in Debug mode the calibration can be very slow.

Let me know if changing to Release worked.

Thanks,

Marek

[Zetee]

Hi,

Did you compile in Debug or Release? When compiling in Debug mode the calibration can be very slow.

Let me know if changing to Release worked.

Thanks,

Marek

Thanks for your reply. I'm sure it was compiled in Release version. Maybe the calibration is too slow in my computer? the server is i7-8700+GTX1080.

[Zetee]

My settings

[MarekKowalski]

Hi,

Sorry, forgot to reply to your previous message. The calibration should be very fast in Release, ~1 second. Your config looks good. Some more things to make sure of:

• Is the marker you are using the one with 0 in it's name?
• Are the depth values for your marker withing the FOV of the depth camera? For now, just to avoid doubt, place the marker directly in front of the Kinect ~1 meter from it.
• Is there some white border around the marker? If not, there could potentially be issues.

If you can, send me screenshot from the client, maybe something will look off.

Thanks,

Marek

[Zetee]

Amazing!! After adding white border around the marker, the calibration succeeded. However, the average FPS in Show live is under 5, is it normal? as the demo says it could runs at 25 FPS.

[MarekKowalski]

That's a bit slow. Can you try disabling filtering in the server settings? Also what is the fps in the client windows (you can see it on the bottom left)?

[Zetee]

I will try that. The client ran at 20-30 fps before connected to server, once calibrated, it dropped to about 4 fps. But saving and reading records seemed fine. By the way, could you provide a more specific doc on how to calibrate multiple Kinect with multiple markers? I think we can get better performance with 4 Azure Kinect from different views. I am not sure if it's alright to calculate transfer matrix using extrinsics of RGB camera. Thanks for your update!

[bustky]

Hey @Zetee how good are the azure kinects quality? does the calibration of a person look better than the old kinects? could you upload a picture of how a person looks on your calibration setup as I was interested in purchasing the new kinects, but due to the price, I wanted to know the increase of quality.

[Zetee]

Hey @bustky Since I haven't used the old kinects, I can't tell you the improvement. But you can get high-quality RGB image (up to 4K), and more detailed depth image (1024x1024, 120°x120°) in this new one. Anyway, I am satisfied with it, and Microsoft even added external synchronization. Here is the image I get from 3 azure kinects. Since the original image is a little large, I convert to jpg format.

[MarekKowalski]

@Zetee , it's weird that the framerate drops after calibration. I'll try to look into it a bit more. In the meantime, could you check if you see a memory leak when you get the slowdown?

As for the multiple marker calibration: I presume you took a look at the manual in the docs section. Is there anything specific that is not clear and I should add?

[Zetee]

@MarekKowalski No memory leak, but CPU usage is above 90%. For the calibration, a camera can only recognize one marker at the same time, so I'm not fully understand how to calibrate with multiple markers when one marker is not visible to all kinects.

[MarekKowalski]

I have some time reserved for looking into this new branch tomorrow, so I'll try to see if I can get a repro of the 90% CPU. Could you also tell me which process does the 90%? Is this the client?

As for multiple markers:

• Each Kinect will only use the largest marker it can see.
• If the largest marker seen by a Kinect is not specified in the settings menu, it will not calibrate.
• Each marker you use has to be specified in the settings menu along with its position and orientation.
• The Kinect determines its position relative to the marker, and subsequently uses the marker's position specified in the settings to establish its position in the world coordinate system. This means that if you do not specify the marker's position in settings correctly, it will not calibrate correctly.
• One way to calibrate using multiple markers is to attach two markers to two sides of a box. You can the specify the rotation as 180 degrees around Y axis, and their distance would be equivalent to the box thickness in Z axis.

Does that make sense? I'll try to update the manual as I keep working on the Azure Kinect version, but it may take some time.

[MarekKowalski]

@Zetee I fixed a memory leak bug today, but unfortunately I was unable to get a repro on the 90% CPU. Could you give me the settings file you are using as well as the exact repro? How many Kinects are you using?

Thanks,

Marek

[Zetee]

@MarekKowalski Thanks for your updating. I think the high CPU usage is that I started multiple clients and server at the same computer. The 3D reconstruction is not my focus now, but I am interested in this topic. I am going to using PCL library for uncalibrated registration if not busy.

[songqiming97]

Hello, may I ask what coordinates are filled in under Bounding box and calibration marekers in calibration mode? In addition, I would like to ask if I can add multiple calibration markers under calibration?

[ChristopherRemde]

The bounding box values are set in meters from the world origin (read: the marker). Min bound values should be negative, Max bound values positive. Just play around a bit with the values and watch the yellow box in the preview, you'll understand it quickly :)

Regarding the markers: yes you can, please take a look here:

https://github.com/Elite-Volumetric-Capture-Sqad/LiveScan3D/blob/development/docs/User%20Guide/Calibration.md