marek-simonik / record3d

Accompanying library for the Record3D iOS app (https://record3d.app/). Allows you to receive RGBD stream from iOS devices with TrueDepth camera(s).
https://record3d.app/
GNU Lesser General Public License v2.1
379 stars 55 forks source link

Setting JPEG compression for in-app captures #52

Closed swtyree closed 1 year ago

swtyree commented 1 year ago

Hi @marek-simonik, thanks for the fantastic app! I use depth cameras for computer vision research, and the ability to capture RGB-D videos without being tethered to a computer is a game-changer.

My one complaint is that the JPEG compression rate for in-app captures leaves a lot of artifacts. Are there any plans to expose that setting in the future? I've made a few captures using the USB interface with lossless RGB streaming, and the resulting images are much better. I realize that a higher setting will significantly increase file sizes on the iPad, but that is something I am willing to work around.

Thanks!

marek-simonik commented 1 year ago

Hi @swtyree and thanks for using Record3D :).

Apart from increased file size (which isn't necessarily a problem, as you mentioned), there is the issue of ensuring real-time performance when processing (nearly) lossless images. I'll take a look into this to see how performance would be affected across different devices under low levels of compression and if it turns out there is a negligible performance drop, I will try to add such option into a future update.

P.S. The lossless USB streaming (USB Streaming RGB quality exactly at 100) uses PNG instead of JPEG, so if your use case requires lossless quality, then I don't think it would be possible to implement this feature in real-time (PNG encoding isn't hardware-accelerated).

swtyree commented 1 year ago

Thanks, @marek-simonik! I think a range of JPEG compression settings would be acceptable. For instance, I tried settings from 70-100 for a frame from a recent lossless capture (cv2.imwrite(filename, img, [cv2.IMWRITE_JPEG_QUALITY, quality])). Beyond 75 or 80, the visual differences were barely perceptible, while the file sizes were noticeably smaller:

quality_070.jpg  28K
quality_075.jpg  31K
quality_080.jpg  36K
quality_085.jpg  43K
quality_090.jpg  56K
quality_095.jpg  86K
quality_100.jpg 188K
quality____.png 428K
marek-simonik commented 1 year ago

OK, if (limited range of) JPEG compression settings is an acceptable compromise for you, then I will add such option into the next Record3D update.

swtyree commented 1 year ago

Thank you! Do you have an estimate for when the next update will be pushed?

marek-simonik commented 1 year ago

The next Record3D update should be on the App Store next week (or earlier).