marek-simonik / record3d

Accompanying library for the Record3D iOS app (https://record3d.app/). Allows you to receive RGBD stream from iOS devices with TrueDepth camera(s).
https://record3d.app/
GNU Lesser General Public License v2.1
383 stars 57 forks source link

Possible to export without LZFSE? #4

Closed andybak closed 4 years ago

andybak commented 4 years ago

I can write a converter but it makes things more of a hassle - especially if I want to create a workflow for non-developers.

Ideally a zip with jpgs for both RGB and depth would be nice.

marek-simonik commented 4 years ago

Hello @andybak,

adding a new export option with JPG visualisation of the depth data seems to me like a very specific request that would not be beneficial to most people, but would instead cause confusion.

I would recommend writing a custom Python script that parses the exported Record3D data and generates the format that you require. To make it very easy for non-developers to run the script, I would recommend generating a standalone executable out of the Python script with all dependencies using e.g. PyInstaller.

The pyliblzfse Python package can be used for decompressing of the LZFSE depth maps.

Feel free to reach out to me if you would want me to help you with the script :)! We can continue our discussion either here or move over to email (support@record3d.app).

andybak commented 4 years ago

Thanks.

Out of curiousity - why not make jpg the default for both? Is it performance or ease of implementation? I can't imagine lossy compression being especially problemmatic for this use case.

marek-simonik commented 4 years ago

I would save the depth data into JPG if I could (it would decrease the file size a lot), but the problem with JPEG is that it is a lossy compression algorithm.

It might seem that having a slightly compressed depth data would not be much of an issue, but the opposite is true; even a slight JPEG compression makes the depth data practically unusable. The depth data is stored in floats and unlike with ordinary color images, JPEG compression is just not acceptable, because the 32bit float values would need to be unpacked into the individual channels of the JPEG image. By performing the lossy JPEG compression on the channel containing MSB bytes, the depth values would change dramatically.

This is why the depth data is being stored without a lossy compression.

andybak commented 4 years ago

Ah. I naively assumed we were using a single grayscale channel for depth but I guess 256 levels of depth is nowhere near enough for the output of your app. I was thinking of the kind of depth channel Facebook 3d photos use but that's a totally different use-case.

marek-simonik commented 4 years ago

That's right, Record3D saves absolute depth values (in meters). I do not use FB, so I did not know about their 3D photos, hence my confusion :).

I am closing this issue, but feel free to comment here or send me an email (support@record3d.app) if you would have any questions.