search

tngh5004 / Omnistitch

This project is the official implementation of our ACM MM 2024 paper, OmniStitch: Depth-aware Stitching Framework for Omnidirectional Vision with Multiple Cameras
4 stars 0 forks source link

About Tensorboard #8

Open DMH517 opened 1 week ago

DMH517 commented 1 week ago

Hi author, I would like to ask something about what is shown in Tensorboard. As shown, are 0/0, 1/0, 2/0, as well as 3/0 images in a batch (by random rotation, channel inversion, etc.)? Also 0/1, 1/1, 2/1, and 3/1 predict which stage of who's optical flow? Tensorboard

DMH517 commented 1 week ago

There is also the problem that in the three images shown in the left half of the above image (respectively, the overlapping image, the label, and the predicted result). The difference between the pairs of to-be-stitched images is very small, with only very little ghost, which is very inconsistent with my preprocessing results. May I ask how you achieved this result by preprocessing the fisheye images? I can't achieve this level of fisheye image correction and therefore the panoramic stitching is not as good as it could be. Looking forward to your reply!

tngh5004 commented 1 week ago

Hi author, I would like to ask something about what is shown in Tensorboard. As shown, are 0/0, 1/0, 2/0, as well as 3/0 images in a batch (by random rotation, channel inversion, etc.)? Also 0/1, 1/1, 2/1, and 3/1 predict which stage of who's optical flow? Tensorboard

  1. As you said, the image pairs of 0/0, 1/0, 2/0, 3/0 have data augmentation of random rotation, channel inversion, etc. It's just a coincidence that they look different in each batch. (It's just randomized).
  2. The optical flow seen in 0/1 through 3/1 is a prediction of the flow between the two input images of the image pair at x/0. Note that the image pair is of the form (img0/groundtruth/img1). And the optical flow map shows the direction of the moving pixels for each color, and the intensity of the flow for each color. You'll need to do a search for this.
tngh5004 commented 1 week ago

There is also the problem that in the three images shown in the left half of the above image (respectively, the overlapping image, the label, and the predicted result). The difference between the pairs of to-be-stitched images is very small, with only very little ghost, which is very inconsistent with my preprocessing results. May I ask how you achieved this result by preprocessing the fisheye images? I can't achieve this level of fisheye image correction and therefore the panoramic stitching is not as good as it could be. Looking forward to your reply!

Please understand that we cannot disclose the unwarping technology because it is our proprietary technology, and when used in the real environment, ghosting is a disadvantage of optical flow base stitching, and it is something that needs to be improved in further research.

DMH517 commented 1 week ago

Hi author, I would like to ask something about what is shown in Tensorboard. As shown, are 0/0, 1/0, 2/0, as well as 3/0 images in a batch (by random rotation, channel inversion, etc.)? Also 0/1, 1/1, 2/1, and 3/1 predict which stage of who's optical flow? Tensorboard

  1. As you said, the image pairs of 0/0, 1/0, 2/0, 3/0 have data augmentation of random rotation, channel inversion, etc. It's just a coincidence that they look different in each batch. (It's just randomized).
  2. The optical flow seen in 0/1 through 3/1 is a prediction of the flow between the two input images of the image pair at x/0. Note that the image pair is of the form (img0/groundtruth/img1). And the optical flow map shows the direction of the moving pixels for each color, and the intensity of the flow for each color. You'll need to do a search for this.

Thanks for the answer, I don't really know much about optical flow technology and as you said I really should brush up on my knowledge. Another confusion I have is that is the optical flow in the Step1 the same thing as the optical flow in the figure above (it feels like a big difference)? image