sniklaus
commented
4 years ago Thank you for your kind words and for bringing this up!
The baseline is 40 constant across all samples. I warped a left image to a right image based on the disparity of the left image and am getting the following results (the first animation shows warped left image versus right image, the second animation shows the left image warped towards the right image).
I used the following code that employs Softmax Splatting to forward warp the left image.
#!/usr/bin/env python
import torch
import cv2
import json
import math
import moviepy
import moviepy.editor
import numpy
import softsplat
##########################################################
fltFov = json.loads(open('00001-meta.json', 'r').read())['fltFov']
fltFocal = 0.5 * 512 * math.tan(math.radians(90.0) - (0.5 * math.radians(fltFov)))
fltBaseline = 40.0
tenImage = torch.FloatTensor(numpy.ascontiguousarray(cv2.imread(filename='00001-bl-image.png', flags=-1).transpose(2, 0, 1)[None, :, :, :].astype(numpy.float32) * (1.0 / 255.0))).cuda()
tenDepth = torch.FloatTensor(numpy.ascontiguousarray(cv2.imread(filename='00001-bl-depth.exr', flags=-1)[:, :, None].transpose(2, 0, 1)[None, :, :, :])).cuda()
tenDisp = (fltFocal * fltBaseline) / tenDepth
tenFlow = torch.cat([-1.0 * tenDisp, 0.0 * tenDisp], 1)
npyWarped = []
for intTime, fltTime in enumerate(numpy.linspace(0.0, 1.0, 11).tolist()):
npyWarped.append((softsplat.FunctionSoftsplat(tenInput=tenImage, tenFlow=tenFlow * fltTime, tenMetric=1.0 + tenDisp, strType='softmax')[0, :, :, :].cpu().numpy().transpose(1, 2, 0) * 255.0).astype(numpy.uint8))
# end
moviepy.editor.ImageSequenceClip(sequence=[npyFrame[:, :, ::-1] for npyFrame in npyWarped + list(reversed(npyWarped))], fps=9).write_gif('warped.gif')
moviepy.editor.ImageSequenceClip(sequence=[npyFrame[:, :, ::-1] for npyFrame in [npyWarped[-1], cv2.imread(filename='00001-br-image.png', flags=-1)]], fps=3).write_gif('compare.gif')Closing for now, please feel free to reopen in case you have any issues.
Hi,
First of all, thank you for providing the wonderful dataset. However, in the provided dataset, there is a json file which describes the FOV only for each 4 views. Is it possible to provide the baseline between each view ?
Thank you!