search

strayrobots / scanner

An app for collecting raw RGB-D scans on iOS devices.
MIT License
31 stars 6 forks source link

Plane Fitting Problem with Depth Data on Stairs #2

Open jungilha opened 9 months ago

jungilha commented 9 months ago

Problem with Depth Data on Stairs When capturing images of stairs using the Strayscanner app, I have observed that when the depth distance exceeds approximately 2.5 meters, the stairs' contours do not appear in the depth data; instead, they are represented as a flat plane. It seems like plane fitting might be occurring during the fusion stage, as discussed in this Apple Developer Forum thread: link. Visualizations Attached for Clarity To better illustrate the issue, I have attached visualizations that utilize RGB, depth, and ARKit data obtained with the strayscanner app. In these visualizations, you can see a 3D reconstruction (with magenta color indicating ARKit position information). When observed from a side view, it becomes apparent that as the depth increases, the stairs appear to become closer to the flat plane. This issue is problematic for my ongoing Pose Estimation project, as the depth data provided by strayscanner is not usable in this situation. Request for Lidar Raw Data Access Option I noticed that starting from iOS 15.4, it is possible to access raw data from the LiDAR sensor, as outlined in this Apple documentation: link. Therefore, I kindly request the addition of an option that allows users to acquire raw data from the LiDAR sensor, which would greatly improve the utility of the strayscanner app for a variety of applications, including projects like mine. Thank you for your attention to this matter.

https://github.com/strayrobots/scanner/assets/141091069/03d8b8c8-e0a2-4832-a7d1-7421ddf2a0e3

https://github.com/strayrobots/scanner/assets/141091069/7e1b50a9-7648-4c51-a513-a418e9e956e1

kekeblom commented 9 months ago

Great catch @jungilha. I haven't looked into the filtering option. As you mention, it wasn't available when I first developed the app. Indeed would be good to add a switch for this or even just have it always disabled, should the output be usable. The general philosophy of the app is anyway to allow the fastest access to raw data for development purposes. I would assume most users would always use the raw option.

If you beat me to it, feel free to submit a pull request for disabling this option. Would be most welcome.

jungilha commented 9 months ago

Thank you for the quick response! @kekeblom. I haven't had any experience with writing Swift code, so I'm not very familiar with it. Adding the option you mentioned would be greatly appreciated as it would be helpful for me. Thank you!

jungilha commented 8 months ago

Hello @kekeblom. I'm reaching out to see if there has been any progress on this issue. Thanks again.

kekeblom commented 8 months ago

Unfortunately, no progress on this. Contributions are welcome. Hope you can find a solution.