aftersomemath
commented
2 years ago Yes, it is possible to do that, but with EVIMO2 there is a better way. The code snippet linked makes the assumption that the pixels were collected from a "far away" viewpoint. With EVIMO2, better results can be achieved by using the per-pixel depth maps in the event camera frame.
The most straightforward way would be something like this:
- Apply the inverse of the event camera's intrinsic calibration to it's per-pixel depth image to get a point cloud in the event camera frame
- Transform the cloud into RGB camera's frame using the vicon pose information
- Apply the RGB camera's intrinsic's to the transformed point cloud
- Sample the RGB image at the resulting points and put the results into the corresponding event camera "color image"
Much of this is implemented in evimo_flow.py. If you adapt this script to render RGB information into the event camera frame we would be happy to accept a PR!
Because the warped points are not guaranteed to fall inside the RGB image there will be pixels in event camera frame without RGB information.
danqu130
Thanks for the opensourced new dataset first and foremost! The new dataset provides higher resolution image and event data, but it was collected with two independent cameras (Samsung and flea), which leads to the misalignment of image and event data. My question is, is it possible to approximate alignment through calibrated matrices similar to DSEC (ref https://github.com/uzh-rpg/DSEC/issues/25#issuecomment-956491275)? This will be of great help in the application of combining two kinds of data.