MarekKowalski / LiveScan3D

LiveScan3D is a system designed for real time 3D reconstruction using multiple Azure Kinect or Kinect v2 depth sensors simultaneously at real time speed.
MIT License
749 stars 202 forks source link

the problem of settings #59

Open NelsonWang1990 opened 2 years ago

NelsonWang1990 commented 2 years ago

Hi~, I am using the Live Scan 3D with 4 Azure Kinect. When I set up the system I have some problems. Could you pls tell what is meaining about orientation and translation in the setting menu.

Thanks~

ChristopherRemde commented 2 years ago

Hey Nelson,

what exactly are the problems you are facing?

The Translation and Rotation settings are just for the markers. You can set a translation and rotation offset for each marker from the origin (origin = first marker).

This is useful if you have for example a calibration cube/box with a marker on each side. The each marker would be rotated 90/180/270 degrees from the first marker, plus the translational offset.

NelsonWang1990 commented 2 years ago

Actually, I am using 4 Azure Kinect as a circle, the sensors are located at the vertices of a rectangle, when I make calibration the result always like this QQ图片20211110105332 I am sure that the markers are correctly setted and the client which are connected to the server are show as calibrated. Is there any mistake I made? Thanks a lot for answer me~

NelsonWang1990 commented 2 years ago

I have another problem now, Could you pls tell what can be the reason of slow FPS. When 5 Azure Kinect are connected the FPS may slow to about 0.5FPS

ChristopherRemde commented 2 years ago

Hey Nelson,

do all cameras (green pivots) show approximately up where they should be in relation to each other? Maybe you can try to move all cameras closer to the marker just to test if they can detect the markers correctly?

And about the FPS: The FPS in the live view can go really low depeding on factors like Network speed, CPU speed ect, but is not really representative of the actual FPS while recording. The capture should have a much higher framerate.

NelsonWang1990 commented 2 years ago

Hey Nelson,

do all cameras (green pivots) show approximately up where they should be in relation to each other? Maybe you can try to move all cameras closer to the marker just to test if they can detect the markers correctly?

And about the FPS: The FPS in the live view can go really low depeding on factors like Network speed, CPU speed ect, but is not really representative of the actual FPS while recording. The capture should have a much higher framerate. Hey, Marek, Thanks a lot for your patient, I will have a try.

NelsonWang1990 commented 2 years ago

Hi,Christopher~ I am doing another experiment, the camera configuration is like this, cut the marker is on the ground at the center of the three cameras. Would you pls tell me how to set the markers, should I add 3 markers or only one marker for calibration. If only 1 marker is needed then the orientation and translation should be what configuration. When I use multi Markers, the calibration resutlt is bad. When I use only one marker, I don't know how to correctly set the markers' orientation and translation

ChristopherRemde commented 2 years ago

Hi Nelson,

If you use only one marker, you should add only one marker. The translation/rotation can be left at zero for all values. The translation/rotation only is only needed for an offset from the first marker.

Image you have two markers printed out, 0 & 1, and now you stick them back to back, so that they are opposite from each other. The first marker has all values set to zero. The second marker has all translation values set to zero, but the rotation on the Y axis is set to 180. If you would now increase the distance between the markers to 1 meter, you should then also set the translational value of the x/y/z axis to 1.

NelsonWang1990 commented 2 years ago

Hi Christopher, Thanks a lot for answering me patiently, I finally understand the calibrition system, the last question is what distance is quite suitble to set the markers? Because in all my experients, the detector doesn’t work stable.

从 Windows 版邮件https://go.microsoft.com/fwlink/?LinkId=550986发送

发件人: Christopher @.> 发送时间: 2021年11月12日 15:03 收件人: @.> 抄送: @.>; @.> 主题: Re: [MarekKowalski/LiveScan3D] the problem of settings (Issue #59)

Hi Nelson,

If you use only one marker, you should add only one marker. The translation/rotation can be left at zero for all values. The translation/rotation only is only needed for an offset from the first marker.

Image you have two markers printed out, 0 & 1, and now you stick them back to back, so that they are opposite from each other. The first marker has all values set to zero. The second marker has all translation values set to zero, but the rotation on the Y axis is set to 180. If you would now increase the distance between the markers to 1 meter, you should then also set the translational value of the x/y/z axis to 1.

― You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/MarekKowalski/LiveScan3D/issues/59#issuecomment-966872280, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AWM4GY5W2V3DVNAW63GL4MDULS34RANCNFSM5HUJA2UQ. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

ChristopherRemde commented 2 years ago

Hi Nelson,

Distance is only one part of the problem while detecting the markers. There are a few tips on how to improve detection:

1: The markers should have a size of at least DinA4, but DinA3 is much better from my experience

2: The print should have a low reflectivity, especially on the black parts.

3: Avoid strong light sources with lots of infrared light (e.g. sunlight) as much as possible during calibration

4: Always check if the marker is visible in both the color and the depth channel

5: Experiement around with the light and rotation of the marker in your space. Sometimes there are reflections of IR light you can't really see, sometimes shuffeling around the marker a bit can help.