How to test flownet2-pytorch???

[jiapei100]

1) Where is checkpoints?

python main.py --inference --model FlowNet2 --save_flow --inference_dataset MpiSintelClean \
--inference_dataset_root ....../mpi-sintel/training/clean/alley_1 \
--resume **/path/to/checkpoints** 

2) Fail to run training

python main.py --batch_size 8 --model FlowNet2 --loss=L1Loss --optimizer=Adam --optimizer_lr=1e-4 \
--training_dataset MpiSintelFinal --training_dataset_root ....../mpi-sintel/training/final/market_2  \
--validation_dataset MpiSintelClean --validation_dataset_root ....../mpi-sintel/training/clean/market_2

Error message:

Parsing Arguments [0.139s] batch_size: 8 [0.140s] crop_size: [256, 256] [0.141s] fp16: False [0.142s] fp16_scale: 1024.0 [0.142s] gradient_clip: None [0.143s] inference: False [0.144s] inference_batch_size: 1 [0.144s] inference_dataset: MpiSintelClean [0.145s] inference_dataset_replicates: 1 [0.146s] inference_dataset_root: ./MPI-Sintel/flow/training [0.146s] inference_n_batches: -1 [0.147s] inference_size: [-1, -1] [0.148s] log_frequency: 1 [0.148s] loss: L1Loss [0.149s] model: FlowNet2 [0.150s] model_batchNorm: False [0.150s] model_div_flow: 20.0 [0.151s] name: run [0.152s] no_cuda: False [0.152s] number_gpus: 1 [0.153s] number_workers: 8 [0.153s] optimizer: Adam [0.154s] optimizer_amsgrad: False [0.155s] optimizer_betas: (0.9, 0.999) [0.155s] optimizer_eps: 1e-08 [0.156s] optimizer_lr: 0.0001 [0.157s] optimizer_weight_decay: 0 [0.157s] render_validation: False [0.158s] resume: [0.159s] rgb_max: 255.0 [0.159s] save: ./work [0.160s] save_flow: False [0.161s] schedule_lr_fraction: 10 [0.161s] schedule_lr_frequency: 0 [0.162s] seed: 1 [0.163s] skip_training: False [0.163s] skip_validation: False [0.164s] start_epoch: 1 [0.165s] total_epochs: 10000 [0.165s] train_n_batches: -1 [0.166s] training_dataset: MpiSintelFinal [0.167s] training_dataset_replicates: 1 [0.167s] training_dataset_root: ....../mpi-sintel/training/final/market_2 [0.168s] validation_dataset: MpiSintelClean [0.169s] validation_dataset_replicates: 1 [0.169s] validation_dataset_root: ....../mpi-sintel/training/clean/market_2 [0.170s] validation_frequency: 5 [0.171s] validation_n_batches: -1 [0.177s] Operation finished

Source Code Current Git Hash: b'ac1602a72f0454f65872126b70665a596fae8009'

Initializing Datasets [0.001s] Operation failed

Traceback (most recent call last): File "main.py", line 139, in train_dataset = args.training_dataset_class(args, True, **tools.kwargs_from_args(args, 'training_dataset')) File "....../flownet2-pytorch/datasets.py", line 112, in init super(MpiSintelFinal, self).init(args, is_cropped = is_cropped, root = root, dstype = 'final', replicates = replicates) File "....../flownet2-pytorch/datasets.py", line 66, in init self.frame_size = frame_utils.read_gen(self.image_list[0][0]).shape IndexError: list index out of range

[TaowzzZ]

I meet the same problem with yours,Could you please tell how did you solve it?

[zhuwanling]

Can you tell me why the args.current_hash = subprocess.check_output(["git", "rev-parse", "HEAD"]).rstrip() will go wrong. FileNotFoundError: [WinError 2] 系统找不到指定的文件。

[toprocker]

@zhuwanling

用Git commit 一下就好了

[Miss-Li97]

How to solve this problem?? Traceback (most recent call last): File "main.py", line 212, in model_and_loss.module.model.load_state_dict(checkpoint['state_dict']) File "/home/lihanyu/Envs/test/lib/python3.6/site-packages/torch/nn/modules/module.py", line 719, in load_state_dict self.class.name, "\n\t".join(error_msgs))) RuntimeError: Error(s) in loading state_dict for FlowNet2: Missing key(s) in state_dict: "flownetc.conv1.0.weight", "flownetc.conv1.0.bias", "flownetc.conv2.0.weight", "flownetc.conv2.0.bias", "flownetc.conv3.0.weight", "flownetc.conv3.0.bias", "flownetc.conv_redir.0.weight", "flownetc.conv_redir.0.bias", "flownetc.conv3_1.0.weight", "flownetc.conv3_1.0.bias", "flownetc.conv4.0.weight", "flownetc.conv4.0.bias", "flownetc.conv4_1.0.weight", "flownetc.conv4_1.0.bias", "flownetc.conv5.0.weight", "flownetc.conv5.0.bias", "flownetc.conv5_1.0.weight", "flownetc.conv5_1.0.bias", "flownetc.conv6.0.weight", "flownetc.conv6.0.bias", "flownetc.conv6_1.0.weight", "flownetc.conv6_1.0.bias", "flownetc.deconv5.0.weight", "flownetc.deconv5.0.bias", "flownetc.deconv4.0.weight", "flownetc.deconv4.0.bias", "flownetc.deconv3.0.weight", "flownetc.deconv3.0.bias", "flownetc.deconv2.0.weight", "flownetc.deconv2.0.bias", "flownetc.predict_flow6.weight", "flownetc.predict_flow6.bias", "flownetc.predict_flow5.weight", "flownetc.predict_flow5.bias", "flownetc.predict_flow4.weight", "flownetc.predict_flow4.bias", "flownetc.predict_flow3.weight", "flownetc.predict_flow3.bias", "flownetc.predict_flow2.weight", "flownetc.predict_flow2.bias", "flownetc.upsampled_flow6_to_5.weight", "flownetc.upsampled_flow6_to_5.bias", "flownetc.upsampled_flow5_to_4.weight", "flownetc.upsampled_flow5_to_4.bias", "flownetc.upsampled_flow4_to_3.weight", "flownetc.upsampled_flow4_to_3.bias", "flownetc.upsampled_flow3_to_2.weight", "flownetc.upsampled_flow3_to_2.bias", "flownets_1.conv1.0.weight", "flownets_1.conv1.0.bias", "flownets_1.conv2.0.weight", "flownets_1.conv2.0.bias", "flownets_1.conv3.0.weight", "flownets_1.conv3.0.bias", "flownets_1.conv3_1.0.weight", "flownets_1.conv3_1.0.bias", "flownets_1.conv4.0.weight", "flownets_1.conv4.0.bias", "flownets_1.conv4_1.0.weight", "flownets_1.conv4_1.0.bias", "flownets_1.conv5.0.weight", "flownets_1.conv5.0.bias", "flownets_1.conv5_1.0.weight", "flownets_1.conv5_1.0.bias", "flownets_1.conv6.0.weight", "flownets_1.conv6.0.bias", "flownets_1.conv6_1.0.weight", "flownets_1.conv6_1.0.bias", "flownets_1.deconv5.0.weight", "flownets_1.deconv5.0.bias", "flownets_1.deconv4.0.weight", "flownets_1.deconv4.0.bias", "flownets_1.deconv3.0.weight", "flownets_1.deconv3.0.bias", "flownets_1.deconv2.0.weight", "flownets_1.deconv2.0.bias", "flownets_1.predict_flow6.weight", "flownets_1.predict_flow6.bias", "flownets_1.predict_flow5.weight", "flownets_1.predict_flow5.bias", "flownets_1.predict_flow4.weight", "flownets_1.predict_flow4.bias", "flownets_1.predict_flow3.weight", "flownets_1.predict_flow3.bias", "flownets_1.predict_flow2.weight", "flownets_1.predict_flow2.bias", "flownets_1.upsampled_flow6_to_5.weight", "flownets_1.upsampled_flow5_to_4.weight", "flownets_1.upsampled_flow4_to_3.weight", "flownets_1.upsampled_flow3_to_2.weight", "flownets_2.conv1.0.weight", "flownets_2.conv1.0.bias", "flownets_2.conv2.0.weight", "flownets_2.conv2.0.bias", "flownets_2.conv3.0.weight", "flownets_2.conv3.0.bias", "flownets_2.conv3_1.0.weight", "flownets_2.conv3_1.0.bias", "flownets_2.conv4.0.weight", "flownets_2.conv4.0.bias", "flownets_2.conv4_1.0.weight", "flownets_2.conv4_1.0.bias", "flownets_2.conv5.0.weight", "flownets_2.conv5.0.bias", "flownets_2.conv5_1.0.weight", "flownets_2.conv5_1.0.bias", "flownets_2.conv6.0.weight", "flownets_2.conv6.0.bias", "flownets_2.conv6_1.0.weight", "flownets_2.conv6_1.0.bias", "flownets_2.deconv5.0.weight", "flownets_2.deconv5.0.bias", "flownets_2.deconv4.0.weight", "flownets_2.deconv4.0.bias", "flownets_2.deconv3.0.weight", "flownets_2.deconv3.0.bias", "flownets_2.deconv2.0.weight", "flownets_2.deconv2.0.bias", "flownets_2.predict_flow6.weight", "flownets_2.predict_flow6.bias", "flownets_2.predict_flow5.weight", "flownets_2.predict_flow5.bias", "flownets_2.predict_flow4.weight", "flownets_2.predict_flow4.bias", "flownets_2.predict_flow3.weight", "flownets_2.predict_flow3.bias", "flownets_2.predict_flow2.weight", "flownets_2.predict_flow2.bias", "flownets_2.upsampled_flow6_to_5.weight", "flownets_2.upsampled_flow5_to_4.weight", "flownets_2.upsampled_flow4_to_3.weight", "flownets_2.upsampled_flow3_to_2.weight", "flownets_d.conv0.0.weight", "flownets_d.conv0.0.bias", "flownets_d.conv1.0.weight", "flownets_d.conv1.0.bias", "flownets_d.conv1_1.0.weight", "flownets_d.conv1_1.0.bias", "flownets_d.conv2.0.weight", "flownets_d.conv2.0.bias", "flownets_d.conv2_1.0.weight", "flownets_d.conv2_1.0.bias", "flownets_d.conv3.0.weight", "flownets_d.conv3.0.bias", "flownets_d.conv3_1.0.weight", "flownets_d.conv3_1.0.bias", "flownets_d.conv4.0.weight", "flownets_d.conv4.0.bias", "flownets_d.conv4_1.0.weight", "flownets_d.conv4_1.0.bias", "flownets_d.conv5.0.weight", "flownets_d.conv5.0.bias", "flownets_d.conv5_1.0.weight", "flownets_d.conv5_1.0.bias", "flownets_d.conv6.0.weight", "flownets_d.conv6.0.bias", "flownets_d.conv6_1.0.weight", "flownets_d.conv6_1.0.bias", "flownets_d.deconv5.0.weight", "flownets_d.deconv5.0.bias", "flownets_d.deconv4.0.weight", "flownets_d.deconv4.0.bias", "flownets_d.deconv3.0.weight", "flownets_d.deconv3.0.bias", "flownets_d.deconv2.0.weight", "flownets_d.deconv2.0.bias", "flownets_d.inter_conv5.0.weight", "flownets_d.inter_conv5.0.bias", "flownets_d.inter_conv4.0.weight", "flownets_d.inter_conv4.0.bias", "flownets_d.inter_conv3.0.weight", "flownets_d.inter_conv3.0.bias", "flownets_d.inter_conv2.0.weight", "flownets_d.inter_conv2.0.bias", "flownets_d.predict_flow6.weight", "flownets_d.predict_flow6.bias", "flownets_d.predict_flow5.weight", "flownets_d.predict_flow5.bias", "flownets_d.predict_flow4.weight", "flownets_d.predict_flow4.bias", "flownets_d.predict_flow3.weight", "flownets_d.predict_flow3.bias", "flownets_d.predict_flow2.weight", "flownets_d.predict_flow2.bias", "flownets_d.upsampled_flow6_to_5.weight", "flownets_d.upsampled_flow6_to_5.bias", "flownets_d.upsampled_flow5_to_4.weight", "flownets_d.upsampled_flow5_to_4.bias", "flownets_d.upsampled_flow4_to_3.weight", "flownets_d.upsampled_flow4_to_3.bias", "flownets_d.upsampled_flow3_to_2.weight", "flownets_d.upsampled_flow3_to_2.bias", "flownetfusion.conv0.0.weight", "flownetfusion.conv0.0.bias", "flownetfusion.conv1.0.weight", "flownetfusion.conv1.0.bias", "flownetfusion.conv1_1.0.weight", "flownetfusion.conv1_1.0.bias", "flownetfusion.conv2.0.weight", "flownetfusion.conv2.0.bias", "flownetfusion.conv2_1.0.weight", "flownetfusion.conv2_1.0.bias", "flownetfusion.deconv1.0.weight", "flownetfusion.deconv1.0.bias", "flownetfusion.deconv0.0.weight", "flownetfusion.deconv0.0.bias", "flownetfusion.inter_conv1.0.weight", "flownetfusion.inter_conv1.0.bias", "flownetfusion.inter_conv0.0.weight", "flownetfusion.inter_conv0.0.bias", "flownetfusion.predict_flow2.weight", "flownetfusion.predict_flow2.bias", "flownetfusion.predict_flow1.weight", "flownetfusion.predict_flow1.bias", "flownetfusion.predict_flow0.weight", "flownetfusion.predict_flow0.bias", "flownetfusion.upsampled_flow2_to_1.weight", "flownetfusion.upsampled_flow2_to_1.bias", "flownetfusion.upsampled_flow1_to_0.weight", "flownetfusion.upsampled_flow1_to_0.bias". Unexpected key(s) in state_dict: "conv1.0.weight", "conv1.0.bias", "conv2.0.weight", "conv2.0.bias", "conv3.0.weight", "conv3.0.bias", "conv_redir.0.weight", "conv_redir.0.bias", "conv3_1.0.weight", "conv3_1.0.bias", "conv4.0.weight", "conv4.0.bias", "conv4_1.0.weight", "conv4_1.0.bias", "conv5.0.weight", "conv5.0.bias", "conv5_1.0.weight", "conv5_1.0.bias", "conv6.0.weight", "conv6.0.bias", "conv6_1.0.weight", "conv6_1.0.bias", "deconv5.0.weight", "deconv5.0.bias", "deconv4.0.weight", "deconv4.0.bias", "deconv3.0.weight", "deconv3.0.bias", "deconv2.0.weight", "deconv2.0.bias", "predict_flow6.weight", "predict_flow6.bias", "predict_flow5.weight", "predict_flow5.bias", "predict_flow4.weight", "predict_flow4.bias", "predict_flow3.weight", "predict_flow3.bias", "predict_flow2.weight", "predict_flow2.bias", "upsampled_flow6_to_5.weight", "upsampled_flow6_to_5.bias", "upsampled_flow5_to_4.weight", "upsampled_flow5_to_4.bias", "upsampled_flow4_to_3.weight", "upsampled_flow4_to_3.bias", "upsampled_flow3_to_2.weight", "upsampled_flow3_to_2.bias".

[leeeeeeo]

python main.py --inference --model FlowNet2 --save_flow --save ./output --inference_dataset ImagesFromFolder --inference_dataset_root ./data/MPI-Sintel-testing/test/clean/wall --resume ./models/FlowNet2_checkpoint.pth.tar

Try this!

[philleer]

@leeeeeeo so what if i want to test my own video? Use the pre-trained model FlowNet2 directly.

[cevan1835]

@leeeeeeo so what if i want to test my own video? Use the pre-trained model FlowNet2 directly.

First of all, you git clone flownet-python and install it. Then import:

import sys sys.path.import('<path to flownet repo>') from models import FlowNet2

Initialize model:

`class Object(object): pass

args = Object() args.rgb_max = 255. args.fp16 = False

model = FlowNet2(args)`

Load state dict:

model.load_state_dict(torch.load('FlowNet2_checkpoint.pth.tar')['state_dict'])

Remember to get ['state_dict']

[poincarelee]

seems the MpiSintel class are not written correctly? this means .flo file and images are both needed, but during inference, why need flo files?

[HaoYongqiang1999]

@leeeeeeo so what if i want to test my own video? Use the pre-trained model FlowNet2 directly.

Hi bro, do you know how to test my own video now? expect for your reply.