Annotating the simulated point cloud data for deep learning tasks

[AJAY31797]

Hi,

I am looking to use Helios++ to develop point cloud data for training deep learning models for 3D object localization.

To apply the data to existing algorithms such as VoxelNet, the data need to be annotated in a certain format, for example, the format of the KITTI dataset. This annotation requires data such as object class, level of occlusion, observation angle, 3D bounding boxes, centroids of bounding boxes, rotation of the object, etc. A lot of this data is relative to the camera position and angle. I wonder if it is possible to get such data from the point clouds generated using Helios++. Could anyone shed some light on it?

Thanks, Ajay

[han16nah]

Hi Ajay,

the idea would be that bringing together your scene definition, the simulated output point cloud and the output trajectory, you should be able to reconstruct a KITTI-like dataset. Here some comments on the data you mentioned:

• object class
  • a) If you input each object as a separate scene part: Use the hitObjectId attribute of the point cloud [preferred option in your case imo]
  • b) If you have several objects per scene part or several scene parts forming one object, you may use material libraries (MTL files) to assign a helios_classification and can then use the Classification attribute of the point cloud.
• Level of occlusion: This depends on your virtual survey configuration and must be calculated on the simulated point cloud (per object/bounding box).
• Observation angle: You can re-calculate the beam vector by using the output trajectory and the output point cloud, which can be aligned via the GpsTime attribute. Then derive the angle(s) from this vector.
• 3D bounding box + centroid: You can derive ground truth 3D bounding boxes of the input 3D scene parts (mesh/voxel models) composing your scene.
• Rotation of the object: I'm not sure how this is defined, but I think also here, you should have your ground truth based on your input 3D scene parts or based on the rotations you may apply to the scene parts in the .scene file (if you apply coordinate transformations there, these must of course also be considered for deriving the bounding boxes).

Cheers, Hannah and Alberto

[AJAY31797]

Hi Hannah and Alberto,

Many thanks for your previous response. I was trying to install Helios++ in my system, but apparently, I am unable to do it. As mentioned on the First Steps page (https://github.com/3dgeo-heidelberg/helios/wiki/First-steps), I unzipped the Helios++ folder and found the helios.exe file in the run subfolder. However, on double-clicking the exe file, I see a dialogue box momentarily and then it automatically gets closed. I am trying to run the software in the correct way. Is there a proper UI available for the software or I need to execute the commands from the command line only? If the UI is available, may I know how can I install the software? I am using a Windows 10 64-bit version.

Best Regards, Ajay

[han16nah]

Dear Ajay,

HELIOS++ is a command-line tool (with Python bindings) and there is no UI. As stated in First steps, please run the software by opening a terminal, navigating to the unzipped HELIOS++ folder and typing run\helios (equivalent to run\helios.exe) followed by the (relative) path of your survey file. E.g., to run one of the demo surveys:

run\helios data\surveys\demo\tls_arbaro_demo.xml

You can add further optional arguments, e.g., if you want the simulated point cloud(s) as LAZ:

run\helios data\surveys\demo\tls_arbaro_demo.xml --lasOutput --zipOutput

You can find the simulated point clouds in the output subfolder.

Best wishes,

Hannah

[AJAY31797]

Thanks for your response.

Best Regards, Ajay