search

microsoft / Azure-Kinect-Sensor-SDK

A cross platform (Linux and Windows) user mode SDK to read data from your Azure Kinect device.
https://Azure.com/Kinect
MIT License
1.5k stars 621 forks source link

Body orientation flip when wearing glasses #575

Open PierrePlantard opened 5 years ago

PierrePlantard commented 5 years ago

Hello, we have an issue with the body tracker when we wearing an hood and protection glasses (see image below). In these case the body constantly flip. I thing this come from the difficulty of the body tracker to well measure the eyes and/or the ears. As you can see on the enclosed video, the skeleton id change often and the skeleton is flipped to 180°. This phenomenon was only found for some people of my team, not for all people. This is a huge issue in industrial or health applications because most people have such protection equipment or hygienic cloths.

To Reproduce

  1. Tested in Simple_3D_Viewer sample
  2. Wear protection glasses or other

Expected behavior The global orientation stability need to be improved. Some ideas :

Screenshots testBTGlass

Desktop (please complete the following information):

Brekel commented 5 years ago

I've seen these kind of flips (as well as the tracking being lost and being picked up as another body ID as can bee seen in your clip) on occasions when not wearing glasses too.

It seems fairly robust on simple poses standing and walking but when crouching, bending, rotating it can be quite fragile. The 360 degree rotation tracking is nice though, as is the quick identification.

cdedmonds commented 5 years ago

Thank you for reporting this issue. We are investigating fixes to make flipping less common.

PierrePlantard commented 5 years ago

Thank you for your response. Keep us in touch.

Best regards,

Pierre

Le mar. 6 août 2019 à 19:23, Chris Edmonds notifications@github.com a écrit :

Thank you for reporting this issue. We are investigating fixes to make flipping less common.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/microsoft/Azure-Kinect-Sensor-SDK/issues/575?email_source=notifications&email_token=AALQ6V5TYDJJLBINAXMBPCDQDGXQPA5CNFSM4II4KJ62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3V3SNQ#issuecomment-518764854, or mute the thread https://github.com/notifications/unsubscribe-auth/AALQ6VZMW3OSIDU47V3T2ITQDGXQPANCNFSM4II4KJ6Q .

RichRH commented 5 years ago

Hi, we've been testing the Azure Kinect for a while now and it seems the biggest issue we're facing is this random flipping (SDK 1.2.0, tracker 0.9.2, 'K4A_DEPTH_MODE_NFOV_UNBINNED', 'K4A_IMAGE_FORMAT_COLOR_BGRA32', 'K4A_COLOR_RESOLUTION_720P').

Due to our use case we either have to mount low (~floor height) or high (~2.3m), tilted up/down to track a person standing ~2.5m away. Our preference is low as it seems to provide better tracking of smaller people/children and fewer issues with perspective/lean.

The problem we're seeing is that in the low position this flip issue is occurring quite frequently, often when just standing still (without glasses). It seems leaning slightly towards the camera (showing it more face) helps but is not a practical fix. In the high position this flipping is far less prevalent, I'd presume down to more face being visible, but instead opens up other issues such as not tracking shorter people and more acute angles to compensate for.

I'm currently looking into trying to catch the flip somehow and filter it out but wondered if you could provide any updates on this issue or any further guidance on how to minimise it?

Cheers, Richard

PierrePlantard commented 4 years ago

Hi, Any update relative to this bug ? It is also the biggest issue we are facing. I can send you some example in video to illustrate the problem (in private). Keep us in touch,

Cheers, Pierre

jesd23 commented 4 years ago

N^ Hot. Pick wisely please.

On Nov 21, 2019, at 8:24 AM, "PierrePlantard" notifications@github.com<mailto:notifications@github.com> wrote:

Hi, Any update relative to this bug ? It is also the biggest issue we are facing. I can send you some example in video to illustrate the problem (in private). Keep us in touch,

Cheers, Pierre

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2FAzure-Kinect-Sensor-SDK%2Fissues%2F575%3Femail_source%3Dnotifications%26email_token%3DAL7WBXEHAEBUDA3CV6TYLCDQU2Y3DA5CNFSM4II4KJ62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEE2ZZLQ%23issuecomment-557161646&data=02%7C01%7C%7Cfc1e18d39e9a4c33b53b08d76e9f42bc%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637099502602927450&sdata=CqgkceOYm2dZfcj8Ck%2FtDCzcWNGaFJj7tyOo6TPd4%2Bg%3D&reserved=0, or unsubscribehttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAL7WBXFSU4DNN6OGVANQ4IDQU2Y3DANCNFSM4II4KJ6Q&data=02%7C01%7C%7Cfc1e18d39e9a4c33b53b08d76e9f42bc%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637099502602937461&sdata=jwmXLrt1Se12eUAClYwMt00rIS2SXw4Yx2jKH5yWU6U%3D&reserved=0.

jesd23 commented 4 years ago

Blu easy to near grn two not one malt .

On Nov 21, 2019, at 8:31 AM, "James Edmond Smith" notifications@github.com<mailto:notifications@github.com> wrote:

N^ Hot. Pick wisely please.

On Nov 21, 2019, at 8:24 AM, "PierrePlantard" notifications@github.com<mailto:notifications@github.commailto:notifications@github.com> wrote:

Hi, Any update relative to this bug ? It is also the biggest issue we are facing. I can send you some example in video to illustrate the problem (in private). Keep us in touch,

Cheers, Pierre

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2FAzure-Kinect-Sensor-SDK%2Fissues%2F575%3Femail_source%3Dnotifications%26email_token%3DAL7WBXEHAEBUDA3CV6TYLCDQU2Y3DA5CNFSM4II4KJ62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEE2ZZLQ%23issuecomment-557161646&data=02%7C01%7C%7Cfc1e18d39e9a4c33b53b08d76e9f42bc%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637099502602927450&sdata=CqgkceOYm2dZfcj8Ck%2FtDCzcWNGaFJj7tyOo6TPd4%2Bg%3D&reserved=0, or unsubscribehttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAL7WBXFSU4DNN6OGVANQ4IDQU2Y3DANCNFSM4II4KJ6Q&data=02%7C01%7C%7Cfc1e18d39e9a4c33b53b08d76e9f42bc%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637099502602937461&sdata=jwmXLrt1Se12eUAClYwMt00rIS2SXw4Yx2jKH5yWU6U%3D&reserved=0.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2FAzure-Kinect-Sensor-SDK%2Fissues%2F575%3Femail_source%3Dnotifications%26email_token%3DAL7WBXANYHRHVDZN4UTL2GLQU2ZWRA5CNFSM4II4KJ62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEE22UXI%23issuecomment-557165149&data=02%7C01%7C%7C3d56a4b3bf024cbad20e08d76ea047aa%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637099506979875861&sdata=UdT8rR1ArtJdzcDjGkZb06kx%2FjrRE9pVqBdLDZxLBkU%3D&reserved=0, or unsubscribehttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAL7WBXFKQJ742VMCY7REAODQU2ZWRANCNFSM4II4KJ6Q&data=02%7C01%7C%7C3d56a4b3bf024cbad20e08d76ea047aa%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637099506979885871&sdata=Qc4v1mSS5FfDDriVx1FBPoyw9sm%2FQMHTzDu8wtGw3bE%3D&reserved=0.

qm13 commented 4 years ago

Could you please send us a recording illustrating the issue.

PierrePlantard commented 4 years ago

Thank you, do you have an email to send you the video ? Pierre

PierrePlantard commented 4 years ago

Dear @qm13, please give me an email to send the video.

Regards, Pierre

qm13 commented 4 years ago

@PierrePlantard we would like to get a recording that demonstrates the problems you are reporting. We require a signed data release agreement along with the recording. Could you provide me with contact details so I can provide you both the agreement and upload instructions?

PierrePlantard commented 4 years ago

@qm13 yes, you can email me at pierre.plantard@moovency.com Thanks

PierrePlantard commented 4 years ago

Dear @qm13, I still haven't received your email, is it normal ?

Chrisbright10 commented 4 years ago

This issue is a big one for me as well, I am not able to adopt this camera at this time because of the body flipping. It doesn't seem to be just isolated to the glasses, (please note issue #1074 ). I have had this issue often while trying to implement this camera in my current project. Could we please prioritize this issue? It is the main reason I can't adopt the Azure Kinect DK camera in my current product (and I would so love to switch over to this).

For the record I am using SDK 1.3.0, tracker 1.0.0, 'K4A_DEPTH_MODE_NFOV_UNBINNED', 'K4A_IMAGE_FORMAT_COLOR_BGRA32', 'K4A_COLOR_RESOLUTION_720P'

Might I ask (without diving too far into the codebase)... how are we currently determining the orientation of the body in body-tracking mode?

Chrisbright10 commented 4 years ago

Is there any update on this issue? If not, who can I pay to help fix this issue faster? This is the only issue preventing me from adopting this camera at this time, and I would really love to.

brontesprocessing commented 3 years ago

I have exactly the same problem - sometimes when I wear glasses the problem occurs very often (every several seconds). But it's not easy to note reproduction steps. I don't see any correlation with the sensor's position, angle, lighting conditions, etc.