search

3dgeo-heidelberg / helios

http://uni-heidelberg.de/helios
Other
194 stars 44 forks source link

Is there any plan to consider non-opaque model in the format of OBJ #269

Open ziyike opened 2 years ago

ziyike commented 2 years ago

Is your feature request related to a problem? Please describe. We are trying to do indoor scene scan simulation with helios++, and ultimately using the simulated point clouds as input data to train some deep learning models. However, there are one type of key feature missing, that's the ability of simulated laser beams passing through or reflect upon transparent objects - mainly windows.

Describe the solution you'd like Maybe add light transmitive procedure similar to dealing with detailed voxels for trees. Consider the transparency (or disolve) parameter in OBJ mtl file.

Describe alternatives you've considered At first, thought the helios_reflectance param could be about whether the laser beam reflects back - tried with no luck, or thought wrong. Then, try to remove all glass components from scene - results not idea for later training. Now, we are trying to sample all transparent parts into point clouds, with significant gaps between points, then feed into the voxel loader, just start trying... Does not seem like the ultimate solution though.

Related issues No

Additional context Like the problem considered in this paper, in modern urban scene, the buildings are covered with huge amounts of glasses - transparent object - which will cause different laser scanning results. In the case of the paper, it is a problem to be tackled, yet on our side of the story, it could be a useful feature to consider.

albertoesmp commented 2 years ago

We think that this feature is a really interesting proposal. We will extend HELIOS++ to support use cases with multiple reflections apart from vegetation. However, we want to work on a general-purpose implementation that covers more cases than windows. Thus, it is going to take more time than a specific-purpose implementation.

If you are interested in collaborating with us, we could use your help. For instance, we would appreciate having you as testers for the new implementation when it comes out. Moreover, if you want to suggest some models of interest for multiple reflections, we would like to check them.

Thank you a lot for opening this issue. We really think it will improve the simulator significantly for multiple scenarios such as indoor scanning, urban scanning (i.e., cities), or water scanning.

ziyike commented 2 years ago

@albertoesmp Thanks for your timely response, and yes, I would love to offer some help. If you are interested, I can provide some architecture models with windows and curtain walls later, might have to go through some scrutiny and data desensitization process first, so it could take some time. Looking forward to this feature being implemented, and surely would like to test it when it comes out.

albertoesmp commented 2 years ago

@ziyike Sure, it would be awesome. Test cases help us to develop much better. Let us know when you have your data ready. There is no hurry, we are in the middle of a sprint trying to finish some updates before the next release, so we neither have too much time right now.

celine-mercier commented 2 years ago

We would be very interested in this feature too, in our case for tree leaf transparency! Cheers

bhoefle-3dgeo commented 2 years ago

We would be very interested in this feature too, in our case for tree leaf transparency! Cheers

Hi Celine. I would be interested how you want to integrate "tree leaf transparency". You can also approach me directly. On single- leaf basis, my state of knowledge was that the transmission is very (very) low and for small-footprints not playing a relevant role. If we speak of "canopy transparency", we already have a solution in HELIOS++: see https://doi.org/10.1016/j.rse.2021.112641

celine-mercier commented 2 years ago

Hello, that's a very interesting paper, thanks. We are indeed thinking about canopy transparency for efficiency reasons. We are working with meshes though. Should we think about voxelizing our tree crown meshes? We could also play with leaf size in different areas, but transparency seemed like the simplest solution.

bhoefle-3dgeo commented 2 years ago

Hi, Celine, we could also have an online meeting to discuss your objective and find out how it can be done in HELIOS++. Just send a direct email to me.