voxblox Check failed: (voxel_offset.array() >= 0).all()

[mikexyl]

Hi, I am trying voxgraph with realsense d435i and orbslam2, and I set tsdf voxel size and truncation distance smaller to fit the indoor lab environment as

tsdf_voxel_size: 0.05
tsdf_integrator:
  truncation_distance: 0.010

and I got this error:

F0827 10:47:46.591811  8047 interpolator_inl.h:209] Check failed: (voxel_offset.array() >= 0).all() 
*** Check failure stack trace: ***
    @     0x7f77c71a543d  google::LogMessage::Fail()
    @     0x7f77c71a7253  google::LogMessage::SendToLog()
    @     0x7f77c71a4fcb  google::LogMessage::Flush()
    @     0x7f77c71a7c3e  google::LogMessageFatal::~LogMessageFatal()
    @           0x48abcd  voxblox::Interpolator<>::getQVector()
    @           0x48af49  voxblox::Interpolator<>::getVoxelsAndQVector()
    @           0x492b69  voxblox::Interpolator<>::getInterpVoxel()
    @           0x49432a  voxblox::transformLayer<>()
    @           0x4bd4e2  cblox::SubmapCollection<>::getProjectedMap()
    @           0x4d1f01  cblox::SubmapMesher::generateCombinedMesh<>()
    @           0x4ce8e8  voxgraph::SubmapVisuals::publishCombinedMesh()
    @           0x45d31c  _ZZN8voxgraph15ThreadingHelper22launchBackgroundThreadIMNS_13SubmapVisualsEFvRKN5cblox16SubmapCollectionINS_14VoxgraphSubmapEEERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERKN3ros9PublisherEEJPS2_RNS_24VoxgraphSubmapCollectionESG_RSI_EEEvOT_DpOT0_ENKUlvE_clEv
    @     0x7f77c44bfc80  (unknown)
    @     0x7f77c49956ba  start_thread
    @     0x7f77c3f2e41d  clone

a check failed in voxblox. Is this a bug, or maybe a voxblox version mismatch, which version are you using by the way?

[mikexyl]

This happens occasionally, only twice so far. without this error, it works like magic

[victorreijgwart]

Hi @LXYYY! We usually run it with the master branch of voxblox, on Ubuntu 18.04. Thanks for including your settings and a screenshot. Regarding your settings, for most of the data I work with, a truncation distance of roughly 3 to 4 times the voxel size gives the smoothest surfaces. This can vary depending on the sensor noise and on how many of the surfaces are perceived at very shallow incidence angles (i.e., with light beams that are almost parallel to the surface). If the truncation distance is set to a value that is smaller than the voxel size, almost all surfaces should be missing, and you might also get numerical problems. This doesn't seem to be the case in the screenshot you shared, though. Could it be that there's a typo in truncation_distance: 0.010? Regarding your error, which branches of voxgraph and voxblox were you using? Then I'll search if we might have changed something recently that could be causing this.

[mikexyl]

that 0.010 should be a typo, but the same error also occurred second time when I set size 0.02 and distance 0.05. Now I am in branch feature/temporal_window of voxblox. if it happened again, I will try to switch back to master. and branch master of voxgraph plus a commit to fix unknown set_unknown_covariance_entries_to_undefined.

[victorreijgwart]

The feature/temporal_window branch for voxblox combines all our latest changes and might not be stable for all use cases yet. Are you using the temporal window features or the projective integrator? If not, I'd definitely go for the master branch. In case you do intend to use any of these new features, you could try it in combination with voxgraph's feature/temporal_window branch. This is the main setup I'm currently using, and I haven't seen the issue you posted before. But especially the voxgraph part is not yet finished, so some parts will be missing or not yet correct (like the TFs).

[mikexyl]

in the beginning, I switched both to temporal_window, just to see if there are some noticable improvement. I think I forgot to switch voxblox back. And could you please give me some hint about the basic idea of the temporal_window work you are doing?

[victorreijgwart]

In the voxgraph version that is currently on master, the submaps are created one after the other - each time starting from an empty submap. In the temporal window version, we use a temporally local map instead, in which the point clouds are integrated and then deintegrated after a certain time. The submaps are then created by copying this local map at a regular interval. The main advantage of this approach is that the temporal map is well suited for local path planning and collision avoidance. Another nice consequence is that this approach makes it possible to create temporally overlapping submaps at no additional cost, by setting the period at which they're copied to a lower value than the temporal window length. When mapping with LiDAR, the field of view and range are usually so high that the submaps always overlap well. But with RGB-D cameras, the vanilla version of voxgraph relies quite a bit on the odometry prior to keep the pose graph together in areas where the robot rapidly observed only new geometry for an extended period.

[victorreijgwart]

Since the integration + deintegration is more expensive than just integration, the temporal window branch also includes a new projective integrator. This integrator can provide a (large) boost in speed when using high-resolution sensors in combination with relatively large voxels. It's also fully deterministic, s.t. integration is exactly reversed by deintegration, and is lock-free.

[mikexyl]

I made it to run temporal window with my setup, after tuning the Tf tree. but optimization of the loop closure contrains is messing up the pose graph, I guess it's still a tf issue, between submap poses and the transformation in loop closure msg sent by my modified orbslam2ros(but it works well with master branch). I made a PR to enable checking of loop_closure/enable.

[victorreijgwart]

Given that the loop closure constraints take relative poses T_t1_t2, it shouldn't be affected by the (global) frame handling and TFs, and since it works well with the master voxgraph branch, I also assume your constraints are good :) Looking up submaps by time is not ready yet in the temporal window version, and might return wrong matches (both for the submaps and T_submap_robot relative poses). It's quite likely that this is causing (at least part of) the issue that you described. I'm currently working on this and it'll probably be fixed in the coming days.

[victorreijgwart]

Adding loop closures should now work again on the temporal_window branch. In case you have time to give it a try, let me know if it does or doesn't work for you.

[mikexyl]

Thanks for noticing me Victor, I will try your latest commit next week after finish my work at hand

[mikexyl]

Hey Victor, is there any branch of voxblox, where projective integrator can integrate directly from depth image? if not, I will write one. Is there any obvious difficulty to support depth image? That feature/projective_integrator_rgbd looks like only added some extra tf for rgbd camera, but still only take point cloud as input.

[victorreijgwart]

Hey @LXYYY, we're planning to add support for depth image messages to the feature/projective_integrator_rgbd branch. Since we personally don't use depth cameras at the moment and have a few big deadlines coming up, we might not get to work on this until later this year though. Aside from time, there should be no obvious practical difficulties or theoretical issues. If you would have time to add it, that would be great and I would be happy to support. If you have any questions along the way, please don't hesitate to ask.

[mikexyl]

Hey @victorreijgwart , if the timestamp in loop closure msg is very recent, that the corresponding submap is still being built in tsdf server not yet sent to voxgraph mapper, the loop closure won't be processed, right? So loop closure generated in slam frontend by matching current and previous frames won't work without some modification? How does your frontend deal with this problem?

[victorreijgwart]

Yes, we encountered the same problem when finishing up another project last week. In the version of voxgraph that's currently on master, very recent loop closure msgs simply got assigned to the current submap. For the new temporal window version, the cleanest solution would probably be to buffer them in the frontend and attach them when new submaps are received. Would you be interested in implementing this? Otherwise I could also take a look later this week. As a temporary workaround for the other project, we simply delayed the loop closure msgs by a fixed time interval. The good news is that the issue we discussed around September 1st is fixed. Loop closures are working well again.

[mikexyl]

ok, I will make a pr tomorrow (hopefully

[victorreijgwart]

Hi @LXYYY, I'll close this issue for now and try to review your PR soon. If you have any further questions, don't hesitate to open a new issue :)