Panoramic photo stitching

[emcauliffe]

Hugin library

A ros package exists for hugin and creating panoramic photos

Get sim module working, then we can map in t265 camera data

[emcauliffe]

@jessicabonanno panorama photo research

Panorama Software - May 7, 2023.pdf

[RutheniumVI]

I have added the hugin package to the repo. Making the changes on the panorama branch for now. To test out the simulation:

1. Build the ROS_WS workspace using catkin_make
2. Launch the simulation. Add some objects into the simulation using the insert tab in Gazebo so that the panoramic stitching has distinct object in them. roslaunch simulation test_world.launch
3. Launch the hugin node. The simulation is using the Kinect camera, can be switched to realsense in the future. The camera data from the kinect is published to /camera/color/image_raw topic, thus this topic will be used for the hugin node to get the images. roslaunch hugin_panorama hugin_panorama.launch image:=/camera/color/image_raw
4. The following rosservice call will be used to take a snapshot of the image being seen on the camera. Basically need to call this service, rotate the rover, then call it again, rotate again, ... until you have taken enough pictures. rosservice call /hugin_panorama/image_saver/save
5. Once all the pictures have been taken, need to call the rosservice to stitch them: rosservice call /hugin_panorama/stitch
6. The stitched image can be found in ROS_WS/src/hugin_panorama/images

More information can be found here

[haashimrehan]

Just tested it with the D435 on the Jetson TX2, it works very well and easy to use

[haashimrehan]

Here is the process I used:

1. Launch the D435 camera which publishes images to /camera/color/image_raw roslaunch realsense2_camera rs_camera.launch
2. Launch the hugin node: roslaunch hugin_panorama hugin_panorama.launch image:=/camera/color/image_raw
3. Save some photos rosservice call /hugin_panorama/image_saver/save
4. Stitch them together rosservice call /hugin_panorama/stitch

Note: The stitching service makes the hugin node print some errors when you initially call it. This doesn't mean it failed, the stitching process took about 1 minute for me stitching 6 photos on the TX2

[emcauliffe]

Science module blocks much of the view of the d435 camera. However, some parts of the environment to the left and right of the module are still visible. Perhaps we can take a cropped view of the camera to do panoramas with?

[haashimrehan]

It's definitely possible but the stitching will take longer as we'll need to take more photos.

Perhaps we can use the other camera to get an unobstructed view? (Logitech one since the endoscope will be in use for science)

Our best bet is to record a bag file with both the cameras anyways so we can try both when we get back to the pits.

[emcauliffe]

merged in with #50. Also have another logitech camera to have a more unobstructed view available