search

brade31919 / radar_depth

Source code of the IROS 2020 paper "Depth Estimation from Monocular Images and Sparse Radar Data"
MIT License
87 stars 13 forks source link

About the pretrained argument #13

Closed pjckoch closed 3 years ago

pjckoch commented 3 years ago

Hi @brade31919 ,

me again :) I'm still very glad you shared this, thanks again!

I have a remark and suggestion about args.pretrained. It is set to True by default:

https://github.com/brade31919/radar_depth/blob/5e6e75772ff379aac65379a50d4042a7c64c869d/utils.py#L65

And it can be changed by running main.py with option --no-pretrain. It is described as option to disable using ImageNet pretrained weights:

https://github.com/brade31919/radar_depth/blob/5e6e75772ff379aac65379a50d4042a7c64c869d/utils.py#L61-L62

However, I think ImageNet pretrained weights are always used, as stage-1 and stage-2 model are created with pretrained=True independent of the argument described above:

https://github.com/brade31919/radar_depth/blob/5e6e75772ff379aac65379a50d4042a7c64c869d/model/multistage_model.py#L29

Instead, I think args.pretrained from above refers to loading a checkpoint of the latefusion model. Because ResNet_multistage is created with args.pretrained as input argument and then in ResNet_multistage:

https://github.com/brade31919/radar_depth/blob/5e6e75772ff379aac65379a50d4042a7c64c869d/model/multistage_model.py#L34-L40

So, I think the name args.pretrained is misleading, perhaps it is better to change it to args.load_checkpoint or something similar. The description in utils.py would need to be changed, too.

Thanks and best regards, Patrick

pjckoch commented 3 years ago

Ok, now I saw that you actually recommend to load the pretrained weights from the latefusion model when training the multistage model. Did you do the same to get the results from the paper. I.e. did you train in two stages, first latefusion model for 20 epochs, then multistage model for 20 epochs?

brade31919 commented 3 years ago

Hi Patrick,

Sorry for the late reply. For the first question, I need to do some recap on my own code a bit...

Regarding the second question, yes I always initialize the multistage model using the "trained" latefusion model. It's quite intuitive if seen from the development process. I started from exploring different fusion methods first, so I had the "trained" latefusion model. Then, we started to try different methods to get rid of the noisy measurements, so we then had the multistage model. Initializing the weights from latefusion model helps to stabilize the training.

Best