How to open the result after scanning by STAC structure sensor recorder

[jenny9217]

Didn't know which software is capable of opening the result

[arthare]

Hi Jenny

All the recorder does is record to a .stac file (which is a losslessly-compressed dump of all the depth data it saw during your recording). You can share it to dropbox or something to get it on your computer, but at that point decoding it is up to you. It does not (presently) do 3d reconstruction.

You can use the sample application here to dump the result to .bmp files if you want to, but otherwise I don't believe there are any big-name pieces of software that will open them.

If you've like, STAC has internal software to do the reconstruction and I could do the reconstruction quickly for you.

[jenny9217]

Ahh,thx a lot i, got it. I thought it could capture something like pointclouds, and didn't notice it was a recording process. I'll give a try .bmp files , thanks for your informations.

[arthare]

You could pretty quickly turn the data into pointclouds. Each frame in the file is the 307200 depth pixels that the structure sensor captures, and you can easily get the millimeters from the sensor that each one represents. If you project them according to the structure sensor's FoV and camera properties|, then you get a point cloud.

[jenny9217]

Hi I try to run the process here you provided but ends up nothing? There is no any bmp files been produced. I think it was processing fine, but i couldn't find the file it produced.

[arthare]

@jenny9217 You need to add the "bmp" command to the command-line you ran.

So you'd do simple_stac_decoder.exe yourFile.stac bmp

Note that there's also an option to dump binary files of the depth-in-millimeters from the program: simple_stac_decoder.exe yourFile.stac bin. This may be more useful if you're trying to build point clouds.

Finally, the easiest thing to do would be to grab a copy of visual studio, open the .sln, and just put some code in main.cpp around line 170, where this code is:

                // you're not in file-write mode (aka the 2nd parameter was not included or was zero).  I'll give you some motivation I guess?
                printf("Decompressed depth at timestamp %.02f.  Do something cool with it!\n", timestamp);
                for (int row = 0; row < 480; row++)
                {
                    for (int col = 0; col < 640; col++)
                    {
                        // fish out the depth pixel here
                        float thisPixelDepth = depthInMillimetersBuffer[row * 640 + col];

                        // it is now up to you, intrepid programmer, to make something of this data.
                        // make a model!  make some measurements!  do whatever!
                    }
                }

[jenny9217]

Hi, when i compile the program it show something below it 1>------ Build started: Project: simple_stac_decoder, Configuration: Debug Win32 ------ 1> main.cpp 1>c:\users\jenny\downloads\stac_sample_decoder-master\stac_sample_decoder-master\simple_stac_decoder\main.cpp(115): warning C4244: 'initializing': conversion from 'uint64_t' to 'unsigned int', possible loss of data 1>c:\users\jenny\downloads\stac_sample_decoder-master\stac_sample_decoder-master\simple_stac_decoder\main.cpp(121): warning C4244: 'argument': conversion from 'uint64_t' to 'std::size_t', possible loss of data 1>c:\users\jenny\downloads\stac_sample_decoder-master\stac_sample_decoder-master\simple_stac_decoder\main.cpp(127): warning C4244: 'argument': conversion from 'uint64_t' to 'std::size_t', possible loss of data 1>c:\users\jenny\downloads\stac_sample_decoder-master\stac_sample_decoder-master\simple_stac_decoder\main.cpp(195): warning C4244: 'argument': conversion from 'const uint64_t' to 'long', possible loss of data 1> bmpWriter.cpp 1> Generating Code... 1> simple_stac_decoder.vcxproj -> C:\Users\jenny\Downloads\stac_sample_decoder-master\stac_sample_decoder-master\lzfse\Debug\simple_stac_decoder.exe 1> simple_stac_decoder.vcxproj -> C:\Users\jenny\Downloads\stac_sample_decoder-master\stac_sample_decoder-master\lzfse\Debug\simple_stac_decoder.pdb (Full PDB) ========== Build: 1 succeeded, 0 failed, 1 up-to-date, 0 skipped ========== Is it the reason why i couldn't successfully generate the .bin file? i do have add "bin" to command line,and still looks the same as the picture i show in the last comment.

[arthare]

@jenny9217 That seems odd...

I just tried \lzfse\Debug>simple_stac_decoder h:\temp\tulips.stac bin on my PC and it successfully output a bunch of .bin files in the same directory.

Your best bet might be to start a debugging session in visual studio and see why it isn't successfully finding the "bin" parameter.

[jenny9217]

Hi, thx a lot again and I finally succeeded. And I got both .bin and .bmp files, and find out .bmp is kind of a depth image? And a bit frustrated since I'm not a coding profession, it's a bit kind of difficult for me to decode(? .binary file to build points clouds. Or maybe could you give me some hints for doing that, and I also found an open source software called Cloudcompare(if you knew, but ends up .bin file couldn't be processed through the software.

[arthare]

Hi @jenny9217

I'll add something to output PLY files today.

[arthare]

Hi @jenny9217

I have updated the code so that it can export .PLY files. If you do a git clone of the repository and recompile, you should be able to do .PLY files.

[jenny9217]

Hi@arthare Thanks a lot!!!!It's really helpful!!!

[jenny9217]

Hi,i'm comming again. Just wondering how did you get the number of 6.5536 as below. And since my devices is ipad air2, wondering is it makes any differences when catching the streams? Thx.

[arthare]

That's because there appeared to be more than 11 bits of information in the depthInMillimeters buffer that the structure sensor returned, so I wanted to stretch it out.

On the ipad, it takes depthInMillimeters (which has a range between 0 and 9999) and multiplies it by 6.5536, which results in a range from 0 to 65535, that fully utilizes the 2-byte unsigned short array that I end up compressing.

So on the PC side, it needs to take that same array of unsigned shorts, and divide them by 6.5536 to get back to the original values that the structure sensor actually returned in depths in millimeters.

On Wed, Jun 28, 2017 at 2:48 AM, jenny9217 notifications@github.com wrote:

Hi,i'm comming again. Just wondering how did you get the number of 6.5536 as below. [image: image] https://user-images.githubusercontent.com/28598818/27622598-ad61a952-5c0a-11e7-8ae9-6c83357d716e.png And since my devices is ipad air2, wondering is it makes any differences when catching the streams? Thx.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/arthare/stac_sample_decoder/issues/1#issuecomment-311572117, or mute the thread https://github.com/notifications/unsubscribe-auth/AByHD9dXzb-yXqoz4wquncT-Y3JsGVnhks5sIfcugaJpZM4NWpvQ .

[jenny9217]

Hi again, just want to know how can i get the sensors exterior orientation during recording ? could the information been seen in the compiling process?? or maybe need some other process so i could be able to get the sensors positions? And wondering does the recording just record the deth images? No RGBs? Really thx for your help.

[arthare]

We do not include the sensor's exterior orientation.

We only record depth images.

On Wed, Sep 13, 2017 at 8:14 AM, jenny9217 notifications@github.com wrote:

Hi again, just want to know how can i get the sensors exterior orientation during recording ? could the information been seen in the compiling process?? or maybe need some other process so i could be able to get the sensors positions? And wondering does the recording just record the deth images? No RGBs? Really thx for your help.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/arthare/stac_sample_decoder/issues/1#issuecomment-329148795, or mute the thread https://github.com/notifications/unsubscribe-auth/AByHDwj58i9GTqr4BoV4FAnXiciSxOxTks5sh8cjgaJpZM4NWpvQ .

[jenny9217]

Hi again, maybe wrong questions,I'm just wondering how did you get the camera position since you could be able to make point clouds as output? Thx a lot.

Sent from my iPhone

On 14 Sep 2017, at 7:13 AM, arthare notifications@github.com<mailto:notifications@github.com> wrote:

We do not include the sensor's exterior orientation.

We only record depth images.

On Wed, Sep 13, 2017 at 8:14 AM, jenny9217 notifications@github.com<mailto:notifications@github.com> wrote:

Hi again, just want to know how can i get the sensors exterior orientation during recording ? could the information been seen in the compiling process?? or maybe need some other process so i could be able to get the sensors positions? And wondering does the recording just record the deth images? No RGBs? Really thx for your help.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/arthare/stac_sample_decoder/issues/1#issuecomment-329148795, or mute the thread https://github.com/notifications/unsubscribe-auth/AByHDwj58i9GTqr4BoV4FAnXiciSxOxTks5sh8cjgaJpZM4NWpvQ .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/arthare/stac_sample_decoder/issues/1#issuecomment-329322926, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AbRiIveXbUcIFnn2cM81WSv2XFsupsSQks5siGFwgaJpZM4NWpvQ.

[arthare]

For our reconstructions, we match each frame of point clouds against previous point clouds. We can detect camera motion just by analyzing the point clouds and comparing to past frames.

On Wed, Sep 13, 2017 at 8:31 PM, jenny9217 notifications@github.com wrote:

Hi again, maybe wrong questions,I'm just wondering how did you get the camera position since you could be able to make point clouds as output? Thx a lot.

Sent from my iPhone

On 14 Sep 2017, at 7:13 AM, arthare <notifications@github.com<mailto: notifications@github.com>> wrote:

We do not include the sensor's exterior orientation.

We only record depth images.

On Wed, Sep 13, 2017 at 8:14 AM, jenny9217 <notifications@github.com< mailto:notifications@github.com>> wrote:

Hi again, just want to know how can i get the sensors exterior orientation during recording ? could the information been seen in the compiling process?? or maybe need some other process so i could be able to get the sensors positions? And wondering does the recording just record the deth images? No RGBs? Really thx for your help.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/arthare/stac_sample_decoder/issues/1# issuecomment-329148795, or mute the thread https://github.com/notifications/unsubscribe-auth/ AByHDwj58i9GTqr4BoV4FAnXiciSxOxTks5sh8cjgaJpZM4NWpvQ .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/ arthare/stac_sample_decoder/issues/1#issuecomment-329322926, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ AbRiIveXbUcIFnn2cM81WSv2XFsupsSQks5siGFwgaJpZM4NWpvQ.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/arthare/stac_sample_decoder/issues/1#issuecomment-329334752, or mute the thread https://github.com/notifications/unsubscribe-auth/AByHD1Cn4BCuug7DdwuqHiavtuq_FouWks5siHPggaJpZM4NWpvQ .