pre-training weight

[yaorong0921]

Hi, the link to the drive now is updated and you may download our trained weights from the drive. If you still cannot have the access to it, pls let me know.

Best,

[Ocean728]

Hi, I'm so glad to have received your response. the trained weights from the drive had been downloaded sucessfully. I cannot find a test file to evaluate 3D-ResNet50 in the repo. and how can I finish this work with "save_best_3DResNet50.pth". Looking forward to your reply.

Sincerely,

[yaorong0921]

Hi, to evaluate it the trained model, you can add two args in the "run-3DResnet.sh": --resume_path /path_to_model/save_best_3DResNet50.pth --no_train

I hope it helps you.

Best,

[Ocean728]

thank you very much. I would like to consult two questions: 1, there is a SyntaxError to the line 1 of the file "run-3DResnet.sh " when I run it, and the error point to python. 2, There seems to be no file to test a single video in this repo，so how to show the performance of the framework？ thank you for you time and patience.

[yaorong0921]

Hi,

1. could you show your error?
2. By performance, do you mean to give a single video and then show the predicted action? Once you have trained two branches for in- and outside videos, you can train the classifier using the features from two branches. Concretely: The features for inside videos F_I are the 2048-dim features before the last fully-connected layer in 3D Resnet. The features for outside F_O videos are the (32112176) output tensors of the last layer of ConvLSTM Encoder.

Once you have the features, you can use the architecture list in TABLE II in the paper to train the classifier: F_O is first put into four Conv-Blocks and then concatenated together with F_I to form F_both. F_both should be a 3072-dim feature. F_both is fed into two fully-connected layers used for making the final decision.

Best,

[Ocean728]

The error is shown in the image below.

[yaorong0921]

Hi, please add a new file named "opts.json" and try to run the code.

Best,

[yaorong0921]

Hallo,

did you use your own annotation files (.csv) files? You could check whether the video_names[i] provides the item like " rchange/20141220_162850_1165_1315/".

Best,

Am Do., 1. Apr. 2021 um 11:13 Uhr schrieb ROBERT @.***>:

there is a error as following when I evaluate this model with the "save_best_3DResNet50.pth". It is necessary to inform that because the dataset is very large, I only use part of the data in the “face_camera” to evaluate this model. Thank you for your explanation. [image: error] https://user-images.githubusercontent.com/73243594/113271735-6a2b1b80-930d-11eb-97cd-047adec72bdd.jpg

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/yaorong0921/Driver-Intention-Prediction/issues/1#issuecomment-811772323, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJJCT4HKP26A3ZBOIAVSL4TTGQ2LZANCNFSM4ZHWNC4A .

[Ocean728]

Hi,

I'm sorry I didn't describe the question clearly. I plan to make an attempt with a part of the data first, and generate my own files(.csv).

1. when I run the "extract_images.py" ,the item like"Output #0, image2, to './face_camera\lchange\20141019_091136_63_213\image-%4d.png'".And when I run the "n_fold_Brain4cars.py", the item like"lchange\20141019_091136_63_213\,lchange\20141019_091136_63_213,126,validation".
2. how can I get the features from two branches?
3. Is there a pre-training weight of ConLSTM here?

Best,

[yaorong0921]

Hi,

1. I think you generated the files in the windows system. Pls replace the '\' by '/' and see whether it works.
2. Once you have trained two branches for in- and outside videos, you can train the classifier using the features from two branches Concretely: The features for inside videos F_I are the 2048-dim features before the last fully-connected layer in 3D Resnet. The features for outside F_O videos are the (32112176) output tensors of the last layer of ConvLSTM Encoder.

Once you have the features, you can use the architecture list in TABLE II in the paper to train the classifier: F_O is first put into four Conv-Blocks and then concatenated together with F_I to form F_both. F_both should be a 3072-dim feature. F_both is fed into two fully-connected layers used for making the final decision.

3. So there is no pretrained weight of ConvLSTM and I trained ConvLSTM using the Optical Flow images from scratch. I uploaded the ConvLSTM model trained with fold0.csv to https://bwstaff-my.sharepoint.com/:f:/g/personal/yao_rong_bwstaff_de/EpmuNb3eB7hPgv2DmeBrQ1ABqgQ6uInXudrpfQQyPgmJZA?e=6LkUKH . You may try it but also train from scratch may be better since we use different fold devisions.

Best,

Am Fr., 9. Apr. 2021 um 11:38 Uhr schrieb ROBERT @.***>:

Hi,

I'm sorry I didn't describe the question clearly. I plan to make an attempt with a part of the data first, and generate my own files(.csv).

1. when I run the "extract_images.py" ,the item like"Output #0, image2, to './face_camera\lchange\20141019_091136_63_213\image-%4d.png'".And when I run the "n_fold_Brain4cars.py", the item like"lchange\20141019_091136_63_213,lchange\20141019_091136_63_213,126,validation".
2. how can I get the features from two branches?
3. Is there a pre-training weight of ConLSTM here?

Best,

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/yaorong0921/Driver-Intention-Prediction/issues/1#issuecomment-816558307, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJJCT4B7UISUM7UABR4QB23TH3DIJANCNFSM4ZHWNC4A .

[Ocean728]

yes.I generated the files in the windows system. I have changed the '\' to '/' by added the code like image below

and then I run the "run-3DResnet.sh"，but the program can not go on，like this image below.

[yaorong0921]

It seems that you did not read any video inputs. Pls check your dataloader e.g. len(train_data) that you have input data.

Am Fr., 9. Apr. 2021 um 14:57 Uhr schrieb ROBERT @.***>:

yes.I generated the files in the windows system. I have changed the '' to '/' by added the code like image below [image: -38fdcdbe20e19529] https://user-images.githubusercontent.com/73243594/114182536-5765a780-9908-11eb-9824-783ea36e9531.jpg

and then I run the "run-3DResnet.sh"，but the program can not go on，like this image below. [image: 6d89f4ec72300ca4] https://user-images.githubusercontent.com/73243594/114183266-1de16c00-9909-11eb-8e2d-b6f1bb1957c0.png

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/yaorong0921/Driver-Intention-Prediction/issues/1#issuecomment-816662897, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJJCT4GAK4KXKOPUNUJCHU3TH32UPANCNFSM4ZHWNC4A .

[Ocean728]

and also all tensors are zero when I train the model with pre-training weight，like this

[yaorong0921]

If you load the pretrained model "resnet-50-kinetics", there will be a message printed out. Pls be careful about all the path when you load in, since os.path.join() behaves not equally on Windows

Am Fr., 9. Apr. 2021 um 15:06 Uhr schrieb ROBERT @.***>:

and also all tensors are zero when I train the model with pre-training weight，like this [image: -5d46d9343b772c23] https://user-images.githubusercontent.com/73243594/114184328-52a1f300-990a-11eb-93b0-ddfb486ac472.png

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/yaorong0921/Driver-Intention-Prediction/issues/1#issuecomment-816668024, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJJCT4H5Z3ASDV4DPIZCTT3TH33T3ANCNFSM4ZHWNC4A .

[Ocean728]

Yes，the message of the pretrained model "resnet-50-kinetics" has been printed out. the error occurred not only at the '\'，but also at the comma separator when I run "n_fold_Brain4cars.py"，as shown in the following image.

[yaorong0921]

This is because Glob() also returns the path with “/“, and to extract the target label, the path needs to find “/“ first: (line 35 in “n_fold_Braun4cars.py”. So what you can try is to change line 35 to: = fbase.find(‘\’, 1)

ROBERT @.***> schrieb am Sa. 10. Apr. 2021 um 13:03:

Yes，the message of the pretrained model "resnet-50-kinetics" has been printed out. the error occurred not only at the ''，but also at the comma separator when I run "n_fold_Brain4cars.py"，as shown in the following image. [image: 1945b9ceadf68b1e] https://user-images.githubusercontent.com/73243594/114267531-d10d9c00-99c1-11eb-898f-33ccde450f75.png

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/yaorong0921/Driver-Intention-Prediction/issues/1#issuecomment-817118367, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJJCT4EPASKAXAWXWPG7TO3TIAV7XANCNFSM4ZHWNC4A .

[Ocean728]

Thank you very much.

The cause of the error may be that the data is not loaded when "run-3DResnet.sh " was run. However, there were not any errors could be found in "opts.json" when I checked all paths. Could you please help me find out what the possible problems are in this repo at your convenience.(The data of face_camera has been deleted in advance) Driver-Intention-Prediction.zip

Esteem it a favor!

[yaorong0921]

Hi, I cheked the error and I thought it could be that the video_path is not correct. In the image, the video_path is "E:/pycharm.... prediction\brain4cars...". Again, this mixed up of '/' or '\' can cause the problem that the dataloader does not load any data. Please check line 116 in Brain4cars.py. Also, please print out the len(dataloader), to make sure there is data loaded.

Best,

Am Mo., 12. Apr. 2021 um 15:16 Uhr schrieb ROBERT @.***

:

Thank you very much.

The cause of the error may be that the data is not loaded when "run-3DResnet.sh " was run. However, there were not any errors could be found in "opts.json" when I checked all paths. Could you please help me find out what the possible problems are in this repo at your convenience.(The data of face_camera has been deleted in advance) Driver-Intention-Prediction.zip https://github.com/yaorong0921/Driver-Intention-Prediction/files/6296938/Driver-Intention-Prediction.zip

Esteem it a favor!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/yaorong0921/Driver-Intention-Prediction/issues/1#issuecomment-817802484, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJJCT4C2O3CXEZ7DCGGNAHLTILXCJANCNFSM4ZHWNC4A .

[Ocean728]

Thank you for your kind help. I checked this repo again,and here are some supplementary descriptions.

1. The video_path could be found,as shown below.( part of face_camera)

And the defined "make_dataset" is nomal,but the dataloader still does not load any data.

2. Maybe,the error happened in the "class Brain4cars_Inside". The"get_default_video_loader" seems to be invalid,and the three parameters,video_dir_path, frame_indices and image_loader in"def video_loader"cannot be found. The "self.data" could be printed out normally,as shown below.

All thoes problems above are encountered when evaluating the "3D-ResNet50" with the "save_best_3DResNet50.pth" in the drive. However,after changing "n_classes" from 400 to 5, the model can be trained with the "save_best_3DResNet50.pth" successfully. So,I wonder why the "save_best_3DResNet50.pth" can be used for training but not directly for evaluation.

There are another two questions：

1. Why is the learning rate 0.0

2. There are only the last five epochs(--epochs 30) evaluated, When evaluating the model with "save_best.pth" trained by myself.

[Sidd1609]

How were you able to train the model using "save_best_3DResNet50.pth"? @Ocean728.

data loader is not working properly.

[Ocean728]

I don't.

---Original--- From: @.> Date: Tue, Mar 28, 2023 02:49 AM To: @.>; Cc: @.**@.>; Subject: Re: [yaorong0921/Driver-Intention-Prediction] pre-training weight(#1)

How were you able to train the model using "save_best_3DResNet50.pth"? @Ocean728.

data loader is not working properly.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>