For Optical Flow, is there any pre trained model? and how I train from scratch?

[hbzhang]

For python geonet_main.py --mode=test_flow --dataset_dir=/path/to/formatted/testdata/ --init_ckpt_file=/path/to/trained/model/ --flownet_type=direct --batch_size=1 --output_dir=/path/to/save/predictions/

Where is the pre-trained model? I know in Google Drive, there are a few pre-trained models. I tried all of them, none of them seem work, all complain with following error:

data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?

If I decide to train from scratch, I need to use the following ( I think), but which data I need to use?

python geonet_main.py --mode=train_flow --dataset_dir=/path/to/formatted/data/ --checkpoint_dir=/path/to/save/ckpts/ --learning_rate=0.0002 --seq_length=3 --flownet_type=direct --max_steps=400000

Thank you!!

[yzcjtr]

1. I have checked the models again and find no problems. Probably you wrongly pass the path name instead of the model name to the init_ckpt_file argument? (i.e. you should use /path_prefix/model rather than /path_prefix/)

2. Please follow the data preparation process, after which you should get the formatted dataset. That's exactly what the dataset_dir should be here.

[hbzhang]

Thanks Zhichao,

1. I know there are several pre trained model in Google drive, which model I should use for opticalflow testing?

2. For training, Kitti has many datasets, which Fraser I should use for training opticalflow?

Thank you!!

On Tue, Jun 5, 2018, 9:10 AM Zhichao Yin notifications@github.com wrote:

1.

I have checked the models again and find no problems. Probably you wrongly pass the path name instead of the model name to the init_ckpt_file argument? (i.e. you should use /path_prefix/model rather than /path_prefix/) 2.

Please follow the data preparation process, after which you should get the formatted dataset. That's exactly what the dataset_dir should be here.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/yzcjtr/GeoNet/issues/9#issuecomment-394702717, or mute the thread https://github.com/notifications/unsubscribe-auth/AAMj1YpyC-BvNYz_Lqu-eXXq-OlK_ePXks5t5oMjgaJpZM4UaTAl .

[yzcjtr]

1. Regarding optical flow learning, we provide pretrained models of DirFlowNet (geonet_dirflownet) and ResFlowNet (geonet_resflownet). They are trained with the same losses but different architectures for ablation study purpose. You can see the paper for details;

2. You simply need to download the raw data. Then format it as described in the Data preparation part of readme,

For flow task, the --dataset_name should be kitti_raw_stereo (--remove_static is left out for learning optical flow in generic cases)

[hbzhang]

Thanks for the clarification, I have tried for a while for this.

For (1) Regarding optical flow learning, we provide pretrained models of DirFlowNet (geonet_dirflownet) and ResFlowNet(geonet_resflownet). They are trained with the same losses but different architectures for ablation study purpose. You can see the paper for details;

I followed the tutorial:

Optical Flow Firstly you need to download the KITTI flow 2015 dataset and its multi-view extension. For replicating our flow results in the paper, a seq_length of 3 is recommended. You need to format the testing data by running

python kitti_eval/generate_multiview_extension.py --dataset_dir=/path/to/data_scene_flow_multiview/ --calib_dir=/path/to/data_scene_flow_calib/ --dump_root=/path/to/formatted/testdata/ --cam_id=02 --seq_length=3 Then you can test your trained model by

python geonet_main.py --mode=test_flow --dataset_dir=/path/to/formatted/testdata/ --init_ckpt_file=/path/to/trained/model/ --flownet_type=direct --batch_size=1 --output_dir=/path/to/save/predictions/

But I am stuck with

--init_ckpt_file=./geonet_dirflownet/model.meta

Note, the model.meta was downloaded from your Google Drive pretrained model. There are three files in geonet_dirflownet directory: model.data-00000-of-00001, model.inde, model.meta

WIth this model, the following script

python geonet_main.py --mode=test_flow complains the error

"data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?"

It looks like the pre-trained model is not correct, or there is something wrong with python kitti_eval/generate_multiview_extension.py, because python geonet_main.py --mode=test_flow uses the data prepared by python kitti_eval/generate_multiview_extension.py.

I do not know where I was wrong....

2) For

You simply need to download the raw data. Then format it as described in the Data preparation part of readme,

I have followed the tutorial for downloading the raw data, the model is able to train (THANKS FOR YOUR EXCELLENT WORK). After training, I put the checkpoint model into

--init_ckpt_file=./checkpoints/model-225000.meta

It complains same error

"data loss: not an sstable (bad magic number): perhaps your file is in a different file format and you need to use a different restore operator?"

I do not know why, but I guess is that for testing, followed your tutorial, I used KITTI flow 2015 dataset and its multi-view extension, but the model trained is based on KITTI raw dataset. Does this cause some problem? I do not know.

3) Also, very related to the second question, I have one puzzle, for your tutorial of optical flow testing, you said:

Firstly you need to download the KITTI flow 2015 dataset and its multi-view extension. For replicating our flow results in the paper, a seq_length of 3 is recommended. You need to format the testing data by running

python kitti_eval/generate_multiview_extension.py --dataset_dir=/path/to/data_scene_flow_multiview/ --calib_dir=/path/to/data_scene_flow_calib/ --dump_root=/path/to/formatted/testdata/ --cam_id=02 --seq_length=3

Then you can test your trained model by

python geonet_main.py --mode=test_flow --dataset_dir=/path/to/formatted/testdata/ --init_ckpt_file=/path/to/trained/model/ --flownet_type=direct --batch_size=1 --output_dir=/path/to/save/predictions/

My question is that why for testing we need KITTI flow 2015 dataset and its multi-view extension, but for training, we only need kitti_raw_stereo? Does this cause some differences?

Thank you again!

[yzcjtr]

1. You should download all the files in the model directory, and set init_ckpt_file=./geonet_dirflownet/model (not model.meta!)

2. The training data following kitti_raw_stereo has excluded the test images in KITTI flow 2015 dataset. For detailed description of the training/test splits, you could see the experimentation part of our paper.

[hbzhang]

Thanks, let me check into it.

Thanks again!

On Tue, Jun 5, 2018 at 1:09 PM, Zhichao Yin notifications@github.com wrote:

1.

You should download all the files in the model directory, and set init_ckpt_file=./geonet_dirflownet/model (not model.meta!) 2.

The training data following kitti_raw_stereo has excluded the test images in KITTI flow 2015 dataset. For detailed description of the training/test splits, you could see the experimentation part of our paper.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/yzcjtr/GeoNet/issues/9#issuecomment-394788015, or mute the thread https://github.com/notifications/unsubscribe-auth/AAMj1WJLWQfzsYMoV-ztjxBZfjwg6Ctkks5t5rtIgaJpZM4UaTAl .

[hbzhang]

Thanks for pointing this out, I changed init_ckpt_file=./geonet_dirflownet/model, the program is able to run 1) python geonet_main.py --mode=test_flow --dataset_dir=/home/cislab315/deeplearning/GeoNet/flowdata/formateddata/ --init_ckpt_file=/home/cislab315/deeplearning/GeoNet/models2/geonet_dirflownet/model --flownet_type=direct --batch_size=1 --output_dir=/home/cislab315/deeplearning/GeoNet/flowdata/predications1/

After finishing the run, for the output_dir=/home/cislab315/deeplearning/GeoNet/flowdata/predications1/, it has a folder called model, under it, it has three sub folders

binary color png

Under color and png, it has many png files, but I opened them, all of them are just empty png file looks like this.

2) Also for optical flow evaluation script,

We again provide evaluation script:

python kitti_eval/eval_flow.py --dataset_dir=/path/to/kitti_stereo_2015/ --pred_dir=/path/to/predictions/

---

pred_dir in this command line is the same as output_dir=/home/cislab315/deeplearning/GeoNet/flowdata/predications1/

I tried, and it complains that it can not find flow_result.txt file.

But if it is not, what it should be?

I assume --dataset_dir is the just the kitti raw stereo data?

Thanks!

[yzcjtr]

1. Visualized flow results should be under color subfolder. The png you showed is a correct flow file in the png subfolder. You could read the code more thoroughly for understanding the format.

2. According to this line, you should be able to infer that output_dir is .../predictions1/model/.

[hbzhang]

Thanks for the answer! Your answer is very clear.

One more thing, I can not seem to find the flow_result.txt file indicated in the following line below.

How is this file generated?

https://github.com/yzcjtr/GeoNet/blob/cf336f6b91a16997d67c5ca0c6666f0b6700c435/kitti_eval/eval_flow.py#L23

Thanks!

On Wed, Jun 6, 2018 at 2:15 AM, Zhichao Yin notifications@github.com wrote:

1.

Visualized flow results should be under color subfolder. The png you showed is a correct flow file in the png subfolder. You could read the code more thoroughly for understanding the format. 2.

According to this line https://github.com/yzcjtr/GeoNet/blob/cf336f6b91a16997d67c5ca0c6666f0b6700c435/kitti_eval/eval_flow.py#L33, you should be able to infer that output_dir is .../predictions1/model/.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/yzcjtr/GeoNet/issues/9#issuecomment-394952583, or mute the thread https://github.com/notifications/unsubscribe-auth/AAMj1epCvVXuMRRawRLwkOm2STiLlEk_ks5t53OMgaJpZM4UaTAl .

[yzcjtr]

The eval_flow.py is exactly the script for generating flow_result.txt

[hbzhang]

Yes, where we go, This line has indicated it

with open(eval_log, 'w') as el:

Thanks again!