search

sejong-rcv / MLPD-Multi-Label-Pedestrian-Detection

[RA-L with IROS2021] Multi-Label Pedestrian Detection in Multispectral data
56 stars 13 forks source link

Questions about training tests #9

Closed wlc-git closed 2 years ago

wlc-git commented 2 years ago

Use the code provided by the author for training, which is obtained MR_all: 98.65 MR_day: 99.52 MR_night: 96.37 Recall_all: 6.53 May I ask what does MR mean? Why is it that recall is getting lower and lower when I retrain? What do these mean respectively and why are they different from the data displayed in your paper MR_all: 7.58 MR_day: 7.96 MR_night: 6.95 Recall_all: 96.70 Different, Is there something I didn't adjust，a little confused, looking forward to your answer, thank you

socome commented 2 years ago

image

Thank you for your interest in our work again! We verified the code again by implementing it in a new docker container. However, there was no error found. It usually takes 20~30 epochs to achieves the similar performance to the result in our paper. Did you run the code in a docker container ? Please, make sure to follow the same environments we provided. Also, MR means the log-averaged miss-rate sampled against FPPI. Please refer to our paper for more detailed information. link

wlc-git commented 2 years ago

Thank you for your reply. After I directly trained with 10 epochs, I saw the results directly during training. I used 7595 pictures after cleaning to train, is this the effect? I haven't changed any parameters, can I only use Docker for testing? If so, could you please share the data set you used in training? Thank you

------------------ 原始邮件 ------------------ 发件人: "sejong-rcv/MLPD-Multi-Label-Pedestrian-Detection" @.>; 发送时间: 2021年11月30日(星期二) 晚上6:07 @.>; @.**@.>; 主题: Re: [sejong-rcv/MLPD-Multi-Label-Pedestrian-Detection] Questions about training tests (Issue #9)

Thank you for your interest in our work again! We verified the code again by implementing it in a new docker container. However, there was no error found. It usually takes 20~30 epochs to achieves the similar performance to the result in our paper. Did you run the code in a docker container ? Please, make sure to follow the same environments we provided. Also, MR means the log-averaged miss-rate sampled against FPPI from the range of [0.01, 1]. Please refer to our paper for more detailed information. link

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

wlc-git commented 2 years ago

Initially MR_all: 98.55

MR_day: 99.57

MR_night: 96.02

Recall_all: 10.24

And then as you train, it gets lower and lower, as you can see here

------------------ 原始邮件 ------------------ 发件人: "sejong-rcv/MLPD-Multi-Label-Pedestrian-Detection" @.>; 发送时间: 2021年11月30日(星期二) 晚上6:07 @.>; @.**@.>; 主题: Re: [sejong-rcv/MLPD-Multi-Label-Pedestrian-Detection] Questions about training tests (Issue #9)

Thank you for your interest in our work again! We verified the code again by implementing it in a new docker container. However, there was no error found. It usually takes 20~30 epochs to achieves the similar performance to the result in our paper. Did you run the code in a docker container ? Please, make sure to follow the same environments we provided. Also, MR means the log-averaged miss-rate sampled against FPPI from the range of [0.01, 1]. Please refer to our paper for more detailed information. link

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

xown3197 commented 2 years ago

Could you show me the terminal log?

wlc-git commented 2 years ago

Is this?

------------------ 原始邮件 ------------------ 发件人: "sejong-rcv/MLPD-Multi-Label-Pedestrian-Detection" @.>; 发送时间: 2021年11月30日(星期二) 晚上7:18 @.>; @.**@.>; 主题: Re: [sejong-rcv/MLPD-Multi-Label-Pedestrian-Detection] Questions about training tests (Issue #9)

Could you show me the terminal log?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

xown3197 commented 2 years ago

I can't see anything.

The training data in the KAIST dataset consists of 25k RGB-Thermal pairs. But you said you used 7541 images for your training data. If you are right about wanting to use the KAIST dataset, I think you should check the data set.

wlc-git commented 2 years ago

Because there are too many samples of the data set, I used the data set cleaned by others. I have used the data set for target detection, and there is no problem. I can't understand that in the training process, loss has been decreasing in my data, but recall has been decreasing, and the false detection rate is getting higher and higher

------------------ 原始邮件 ------------------ 发件人: "sejong-rcv/MLPD-Multi-Label-Pedestrian-Detection" @.>; 发送时间: 2021年11月30日(星期二) 晚上8:01 @.>; @.**@.>; 主题: Re: [sejong-rcv/MLPD-Multi-Label-Pedestrian-Detection] Questions about training tests (Issue #9)

I can't see anything.

The training data in the KAIST dataset consists of 25k RGB-Thermal pairs. But you said you used 7541 images for your training data. If you are right about wanting to use the KAIST dataset, I think you should check the data set.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

rgw117 commented 2 years ago

I am not sure what you mean by " too many samples of the data set". The KAIST dataset consists of 12 subsets ranging from 0 to 11, and 0 ~ 5 subsets are used for training whereas 6~11 subsets are used for inference. The number of training dataset cannot be 7,595 but should be a lot more. You can refer to this website for more information of the dataset.

unizard commented 2 years ago

@wlc-git Please, follow the evaluation protocol [1] and solve your problem by yourself.

[1] Multispectral Pedestrian Detection: Benchmark Dataset and Baselines, CVPR 2015.

wlc-git commented 2 years ago

Ok, thank you

------------------ 原始邮件 ------------------ 发件人: "sejong-rcv/MLPD-Multi-Label-Pedestrian-Detection" @.>; 发送时间: 2021年11月30日(星期二) 晚上8:36 @.>; @.**@.>; 主题: Re: [sejong-rcv/MLPD-Multi-Label-Pedestrian-Detection] Questions about training tests (Issue #9)

@wlc-git Please, follow the evaluation protocol [1] and solve your problem by yourself.

[1] Multispectral Pedestrian Detection: Benchmark Dataset and Baselines, CVPR 2015.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.