Exctract TSDF/ESDF Map from voxgraph

[SchmiederX]

Hey,

Is there a way to extract the ESDF or TSDF Map that voxblox creates in the background? I need a voxelized representation of the whole map so that I can convert it back to a pointcloud for comparison with another algorithm.

Or is there a way to get the map as a pointcloud bevor the Meshing is applied?

Thank you in advance.

[alexmillane]

You want the voxelized map (not surface pointcloud) of all submaps fused together correct?

At the moment there's no service call to do exactly that, but such a map is created when you request the mesh of the combined map. You would just have to make a small modification to save that map to disk.

[SchmiederX]

What do you mean with surface pointcloud ?

Ah perfect, you mean the function where i request, that the map is being saved to a file ?

Thank you for that fast answer!! :)

[alexmillane]

I mean you want the voxels right, not a mesh/pointcloud of the surface? You want freespace voxels for example?

In this function here

https://github.com/ethz-asl/voxgraph/blob/master/voxgraph/src/frontend/voxgraph_mapper.cpp#L435

which generates the pointcloud of the whole map, internally it fuses all submaps to create a single voxelized representation through a call to

https://github.com/ethz-asl/cblox/blob/master/cblox/include/cblox/core/submap_collection.h#L161

you could set up a service call to fuse all the submaps, and save the resulting map to file.

[SchmiederX]

Yes thats exactly what I need. Thank you.

The combined Map is of type 'TsdfMap' , any idea how I could save it so that it is in a known fileformat?

just found this function:

https://github.com/ethz-asl/cblox/blob/927c4d53e547da74f6a4b67f7dba3b87d1c42115/cblox/include/cblox/core/submap_collection.h#L150

that seems to save the submap collection.

[alexmillane]

Yep. The TsdfMap contains a Layer<TsdfVoxel> object which has a member function to save it to file:

https://github.com/ethz-asl/voxblox/blob/master/voxblox/include/voxblox/core/layer.h#L270

So in a few lines of code you should be able to achieve what you want.

Now the file format is protobuf stream containing voxel blocks... You will probably have to do a little more work to get it in whatever format you have your comparison maps in.

[SchmiederX]

Perfect!! Thank you very much for your fast help!

As long as I can understand the file format - and for protobuf there should be a documentation - I might be able to convert it!

[SchmiederX]

Okay I have to reopen it :D

I extracted the protobuf file and started getting into the proto stuff. Now I realized that I need the 'encoding' . If im correct I need to have a definition of the message the stream is encoded in, to create a parser for python/c++. Where is this file located? I found this: https://github.com/ethz-asl/voxblox/tree/master/voxblox/proto/voxblox

and created the respective python files but while parsing the extracted voxel file I get:

google.protobuf.message.DecodeError: Error parsing message

Is there something else I have to take care of ?

[SchmiederX]

Okay further update:

I can 'decompress' the protobuf data now. For anyone interested, the original python library didnt work for me, you need to stream the data and I found this library that worked perfectly: https://github.com/cartoonist/pystream-protobuf

Using the Block definition found here: https://github.com/ethz-asl/voxblox/tree/master/voxblox/proto/voxblox

Now I have a list of Blocks. My question is now, how can I understand this block? My guess is that a Block is a cube of voxels with voxel_size as edge length and voxels_per_side is the number of voxels per side (sounds logical?! :D ) But besides those two parameters there is also the voxel_data param. If the voxels_per_side describe a cube, than the voxel_data maybe describes the xyz coordinate of the mapped point in each voxel of the cube? Because len(voxel_data)/(voxel_per_side * 3 ) = 3

How can I stitch them together to form the complete map?

[alexmillane]

So voxblox provides methods to load layers stored in protobuf files in c++ code:

https://github.com/ethz-asl/voxblox/blob/7f7892de15c7760ce0fd777315fd1f74f73eda02/voxblox/include/voxblox/io/layer_io.h#L56

But I guess you want this in python so maybe your way is better…

I have personal code I have used in the past to load blocks from a file and put them into a dense grid (map) for matlab. Maybe I can just link you to this code and that would help you?

[alexmillane]

The best reference for understanding blocks vs. voxels and how they are arranged is the paper where “voxel hashing” was introduced:

https://niessnerlab.org/papers/2013/4hashing/niessner2013hashing.pdf

[SchmiederX]

Hey Alex,

I tried understanding the block concept a little and came up with a solution where I take the center coordinate and create a cube of 16x16x16 around it using the voxel_size parameter. (0.25 in this case) Thus I created a Pointcloud with every block (455 in my case) having 16x16x16 (4096 ) Points. This is my intermediate result. The rough shape fits the pointcloud I want to compare it to.

Now I have to get rid of all the points that I dont need or that dont have data. As I understand, the voxel_data now has 3 bytes for each voxel inside a block. Based on this function I found inside voxblox:

https://github.com/ethz-asl/voxblox/blob/7f7892de15c7760ce0fd777315fd1f74f73eda02/voxblox/src/core/block.cc#L66

I tried to 'deserialize' this data. But I cant get the correct values as the rgb data in byte 3 for example looks completely different and is mostly empty (0) .

I would really appreciate the Matlab code as I might be able to recreate that in Python. If anyone else needs this type of functionality I might be able to share it somehow :)

[alexmillane]

This function here:

https://github.com/alexmillane/cartographer_cblox/blob/18896e37a552c2fb9c32fcb68336b65c04e3a9cd/src/library/cblox_matlab_converter.cpp#L304

Iterates over all observed voxels in a map and returns their distance values and global indices. I guess you wanna do something similar?

[SchmiederX]

Hey Alex,

basically yes, I want to display (and save) the voxelmap as a pointcloud. Basically I want to create a pointcloud from the TSDF Map. The proto file I created includes blocks if TSDF voxels. As described above I can read them in Manually and display a point for each voxel in a block (4096, because the Block has a dimention of 16x16x16) I now need to decrypt the voxel_data, to remove the obsolete points in that pointcloud. the voxel_data for each voxel includes 3 bytes based of this function: https://github.com/ethz-asl/voxblox/blob/7f7892de15c7760ce0fd777315fd1f74f73eda02/voxblox/src/core/block.cc#L66

what does distance and weight mean (they are pretty large numbers if they have a value)? And the color byte is empty in my extracted files.

Maybe the approach im trying to do doesnt work. Im currently trying to implement the same algorithm in C++ using some code i gathered from voxblox and the cartographer_cblox repo. I couldnt get the cartographer_cblox to run just yet as I have to install cartographer first - I guess :D

Your help is highly appreciated!!

[SchmiederX]

Finally! :D

I got a result I can at least interpret and work more with.

Using the cartographer_cblox as a base I tried to read the tsdf map file I created. As the function only takes a cblox_submap I had to include this function from voxgraph: https://github.com/ethz-asl/voxgraph/blob/05a5e19c8e3336fda2a0bd471c52ea1a7d248edd/voxgraph/src/tools/evaluation/map_evaluation.cpp#L13

to read my created tsdf proto file correctly and write it out as the pointcloud you can see above. I now can use this pointcloud representation to calculate the 'original' pointcloud !

Thank you Alex for all your help!

[alexmillane]

Great to see that you got it working in the end!

I hope that you get some good results using voxgraph!