How to use the differents scripts

[ggaspard54]

Hello,

I'm novice and i'm trying to use the differents scripts.

I manage to use recorder_console to download et process files but I don't understand how to use the other scripts.

Does anyone have a procedure for using others?

thanks in advance

[kianwei96]

hi ggaspard54, sorry that i'm not really answering your question here, but would like to check with you, whether you encountered this issue when you used recorder_console!

running: python .\StreamRecorderConverter\recorder_console.py --workspace_path .\test\ --dev_portal_address :10080 --dev_portal_username @outlook.com --dev_portal_password

gives me this error: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond>

surprisingly, I've tried to download files directly off the portal, with the save icon, and i'm able to download data from my personal apps, but it doesn't seem to download for the recorder app.

appreciate any comments/suggestions! :) hope you get an answer to your question soon!

[fbogo]

@ggaspard54 , the scripts are just examples showing how to process the data... The recorder console already uses a number of them. The readme (https://github.com/microsoft/HoloLens2ForCV/blob/main/Samples/StreamRecorder/README.md) tries to describe their usage.

@kianwei96, are you connected via USB or Wi-Fi? perhaps you might need to update the IP address used?

[kianwei96]

@fbogo thanks for the suggestion! I unchecked SSL in the device portal and it works now after some fiddling. Probably not the safest, but I guess it's ok for now.

I did a test recording for long-throw depth, but am trying to figure out the difference between the pgm file with and with _ab suffix (eg. 132469884363655149.pgm vs 132469884363655149_ab.pgm). Is there existing documentation for how to interpret the saved files?

Thanks!

[fbogo]

The '_ab' suffix stays for 'active brightness' image. For a description of Long Throw (and other sensors) payload, you can have a look at the doc https://github.com/microsoft/HoloLens2ForCV/blob/main/Docs/ResearchMode-ApiDoc.pdf (for ab, see e.g. pag. 11).

[ggaspard54]

@fbogo Oh Okay.. Thanks for your answer.

Is there existing Visual Studio solution or software for interpret the saved files ?

Thanks !

[fbogo]

Hi @ggaspard54 , do you mean to use the images saved by the recorder? The python scripts should give you a few examples about how to use them (e.g. reconstruct point clouds, get hand tracking joints etc.).

[ggaspard54]

Hi @fbogo , Yes I want to reconstruct point clouds (like p31 in ECCV2020-ResearchMode-Repository) and get hand tracking joints. That's my main requirement. This in order to show what's HoloLens 2 see.

I have never done that. I need to create another python script or I need to create a Visual Studio Project ( C++ Console ... other than that?) ?

Thanks !

[kianwei96]

@ggaspard54 if you've gotten the depth information already, you can get your point cloud in .ply format from within the recorder_console by calling process X, or you can selectively process for point cloud only by running the save_pclouds.py script directly like this:

python save_pclouds.py --recording_path /path/to/folder/here

@fbogo thank you for addressing our queries. I have two quick questions regarding interpreting the depth data:

please refer to this album for screenshots! https://imgur.com/a/DZtWMCP

1. I wrote a quick script to inspect the outputs, and

When assuming little-endian byte order, I get pixel values ranging up to ~65000 (2^16), which seems fine after consulting the source code. However, the image that I'm getting has some weird depth discontinuities along surfaces. This exists in both Long Throw and AHAT recordings.

When assuming big-endian byte order, the depth continuity looks sane, but the pixel values now range up to ~1000, which is slightly puzzling to me. Am I doing something wrong here, or is one of them to be expected?

2. The second question is regarding how to interpret the values received from this method.

winrt::check_hresult(pDepthFrame->GetBuffer(&pDepth, &outDepthBufferCount));

Can I, from the values returned here, compute the actual distance/time taken for the IR ray to bounce to and fro? In other words, does a value of 100 in one depth image correspond to the same physical distance as a vlaue of 100 in a separate recording?

Thanks a ton!

[fbogo]

Hi @kianwei96 , A value of max 1000 (1055) shouldn't be surprising if you're looking at AHAT. Note that depth in AHAT mode has some peculiarities, namely it is "aliased depth" (see #12 , or pg. 2 of the tech report). To be sure you're parsing the values correctly, you can also compare the results you get with the one we have in 'save_pclouds.py', where we load the images with OpenCV imread.

To your question 2: In AHAT mode, given the aliasing / wrap around characteristic, a value of 100 does not necessarily correspond to the same physical distance.

[kianwei96]

@fbogo

Is there a difference in depth map quality between AHAT and Long Throw, apart from the difference in frequency and the aliased depth? if there's no difference I think I'll go with LT for the most part.

Regarding Q2, can I confirm that LT mode's readings have stable physical correspondence?

Thanks for the help as always :)

[fbogo]

Yes, LT mode readings should have "physical correspondence" -- if this means, there is no the wrapping around like in AHAT. LT mode should support distances between 200mm and 7500mm (differently from AHAT).