search

lliuz / ARFlow

The official PyTorch implementation of the paper "Learning by Analogy: Reliable Supervision from Transformations for Unsupervised Optical Flow Estimation".
MIT License
249 stars 49 forks source link

Settings of pre-trained models #15

Closed playerkk closed 3 years ago

playerkk commented 3 years ago

First of all, thank you for sharing the code. Awesome work!

I was wondering if you could explain the settings of different pre-trained models provided under the checkpoints folder. For example, there are three models for the Sintel dataset. I assume pwclite_raw.tar comes from pre-training on Sintel raw videos without AR. But how about pwclite_ar.tar? In the paper, it is mentioned that AR is not used for pre-training. Any clarification will be appreciated. Thanks.

lliuz commented 3 years ago

the model named with _ar means training with AR and with a basic pretrained model. for example, pwclite_ar.tar is trained on Sintel training set with pwclite_raw.tar as the pretrained model(which is trained from scratch on raw videos and not used AR in the pre-training stage).

playerkk commented 3 years ago

Thank you for the reply.

I tested Sintel's pwclite_ar.tar on the training set. The results look much worse than the numbers reported in Table 1 in the paper. If I understand correctly, pwclite_ar.tar is the one fine-tuned on the training set and should match the one used in Table 1. Did I miss something? Thank you.

lliuz commented 3 years ago

Can you tell me what result did you get? I run the command 

python3 train.py -c configs/sintel_ft.json -m checkpoints/Sintel/pwclite_ar.tar -e

and get the results same as the paper

* Epoch 1 EPE_0: 2.79 EPE_1: 3.73
playerkk commented 3 years ago

What I got are * Epoch 1 EPE_0: 4.20 EPE_1: 5.19.

I am using PyTorch 1.6.0. Not sure if this causes the difference.

lliuz commented 3 years ago

Hi @playerkk, I tested it in PyTorch 1.6 and found that the problem was caused by the default behavior of align_corners for grid_sample has changed to False since PyTorch 1.3.0.

Specify align_corners=True in utils/warp_utils.py and you will get the correct results.

playerkk commented 3 years ago

After fixing the issue, I can reproduce the numbers. Thanks for the help.

RokiaAbdeen commented 3 years ago

I got these two numbers : 3.20 and 4.23 not the same as yours, when I tried to add align_corners=True I got this error im1_recons = nn.functional.grid_sample(x, v_grid, mode=mode, padding_mode=pad,align_corners=True) TypeError: grid_sample() got an unexpected keyword argument 'align_corners