mathmanu
commented
6 years ago Sparse resampling using maxpool will not be correct in the case of sparse flow for negative values. So I introduced an adaptive_minmax_pool2d() that takes care of negative values. With this I am able to see negative values are also learned properly in the case of sparse GT of KITTI.
def adaptive_minmax_pool2d(input, size): maskp = (input>=0).float() maskn = (input<0).float() output = nn.functional.adaptive_max_pool2d(input maskp, size) - \ nn.functional.adaptive_max_pool2d(-input maskn, size) return output
def multiscaleEPE(network_output, target_flow, weights=None, sparse=False): def one_scale(output, target, sparse):
b, c, h, w = output.size()
if sparse:
#target_scaled = nn.functional.adaptive_max_pool2d(target, (h, w))
target_scaled = adaptive_minmax_pool2d(target, (h, w))
else:
target_scaled = nn.functional.adaptive_avg_pool2d(target, (h, w))
return EPE(output, target_scaled, sparse=sparse, mean=True)
if type(network_output) not in [tuple, list]:
network_output = [network_output]
if weights is None:
weights = [0.005,0.01,0.02,0.08,0.32] # as in original article
assert(len(weights) == len(network_output))
loss = 0
for output, weight in zip(network_output, weights):
loss += weight * one_scale(output, target_flow, sparse)
return loss
ClementPinard
Thank you for this simple and readable code. I am also glad that this works with python 3.5 pytorch 0.3. I look forward to you adding other networks such as FlowNetC and FlowNet2.0. Also other metrics such as percentage outliers would be a great addition.
KITTI flow GT is sparse, but this is not considered in flow reading or in training. I suggest the following changes..
In KITTI.py, the ground truth flow reading is not correct. By looking at the KITTI flow reading script and the readme.txt there, this is what I wrote.
def load_flow_from_png(png_path):
Apart from the above function, the sparse flag has to be passed into several functions. I added a flag called sparse_gt
if args.sparse_gt is None: args.sparse_gt = ('KITTI' in args.dataset)
and this flag is passed to all the relevant functions such as: multiscaleEPE, one_scale, realELE, EPE etc.
With these changes, I am getting more meaningfull EPE values.
Kindly fix this issue.