search

gjy3035 / C-3-Framework

An open-source PyTorch code for crowd counting
MIT License
704 stars 200 forks source link

Combine multiple dataset to train #5

Open wait1988 opened 5 years ago

wait1988 commented 5 years ago

Thanks for sharing your code.It makes life easier.I have one question that will it improve the performance if I combine multiple dataset to train the model?

wait1988 commented 5 years ago

Why do you blur the we dataset outside of the ROI? @gjy3035

gjy3035 commented 5 years ago

@wait1988

  1. we do not conduct the experiment on the combined dataset. I am looking forward to your results!
  2. we find the some patterns outsides of ROI are similar to that of ROI, which results in difficult training. The other motivation is that this scheme make the counter network to distinguish the ROI and the blurred regions.
wait1988 commented 5 years ago

Can you provide the pretrained CSRNet model on GCC dataset?I want to finetune the model on other datasets.Thanks. @gjy3035

wait1988 commented 5 years ago

Besides,what's the LabelNormalize for?

gjy3035 commented 5 years ago

@wait1988

  1. You can download the pre-trained CSRNet model on GCC from this link. In fact, I implement the relative expreimrnts on UCF-QNRF, Shanghai Tech A and B. The concrete results are shown as below: TIM截图20190326112325 I hope you can share your results on other datasets.

  2. For the LabelNormalize, it enlarges the value for each pixel in density map. I find this tricks can effectively improve the performance. I plan to write a technical report to explain the tricks and the implementation details in May, 2019.

wait1988 commented 5 years ago

To be honest,your repo is my first step in crowd counting.So currently I'm just trying to reproduce your reported results.Would you please share your config in UCF_QNRF with CSRNet?That would be very helpful. @gjy3035

wait1988 commented 5 years ago

I use your GCC pretrained model,but error occurs:

self.net.load_state_dict(torch.load(cfg.PRE_GCC_MODEL)) File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 769, in load_state_dict self.class.name, "\n\t".join(error_msgs))) RuntimeError: Error(s) in loading state_dict for CrowdCounter: Missing key(s) in state_dict: "CCN.frontend.0.bias", "CCN.frontend.0.weight", "CCN.frontend.2.bias", "CCN.frontend.2.weight", "CCN.frontend.5.bias", "CCN.frontend.5.weight", "CCN.frontend.7.bias", "CCN.frontend.7.weight", "CCN.frontend.10.bias", "CCN.frontend.10.weight", "CCN.frontend.12.bias", "CCN.frontend.12.weight", "CCN.frontend.14.bias", "CCN.frontend.14.weight", "CCN.frontend.17.bias", "CCN.frontend.17.weight", "CCN.frontend.19.bias", "CCN.frontend.19.weight", "CCN.frontend.21.bias", "CCN.frontend.21.weight", "CCN.backend.0.bias", "CCN.backend.0.weight", "CCN.backend.2.bias", "CCN.backend.2.weight", "CCN.backend.4.bias", "CCN.backend.4.weight", "CCN.backend.6.bias", "CCN.backend.6.weight", "CCN.backend.8.bias", "CCN.backend.8.weight", "CCN.backend.10.bias", "CCN.backend.10.weight", "CCN.output_layer.bias", "CCN.output_layer.weight".

gjy3035 commented 5 years ago
  1. The setting of CSR on QNRF is same as the setting on Shanghai Tech A.
  2. Please check your CSR's module list. You should rename the key values to match your modules' names.
gjy3035 commented 5 years ago

I just run the code to train CSR based on pretrained GCC model, which works well. I think you may use single GPU to train the model. However, our provided model is trained using multi GPUs, which causes that the key value contains '.module'. You should rename the provided pretrained model's keys.

wait1988 commented 5 years ago

Ok,I got it.I think crowd counting is kind of like semantic segmentation with the architecture encoder-decoder.

gjy3035 commented 5 years ago

Semantic segmentation and crowd counting are pixel-wise task. SS is classification and CC is regression. Their network architecture is similar. However, the SOTA SS network maybe perform poorly when directly applying them to CC.

wait1988 commented 5 years ago

@gjy3035 I can get the reported result on QNRF with CSRNet-ImgNt.Then I use pretrained GCC model to finetune,with the same configuration,I can't get better result.So do I need to freeze front end layers and decrease the learning rate?

gjy3035 commented 5 years ago

I think you should check the weights is successfully loaded before training. The setting of the two experiments are same. TIM截图20190329142517

wait1988 commented 5 years ago

162 epoch,best mae:120.6,best mse:195.6 for now.I'll keep training to see if I can get your result.

homurajiang commented 5 years ago

I see your anwser of this

"I just run the code to train CSR based on pretrained GCC model, which works well. I think you may use single GPU to train the model. However, our provided model is trained using multi GPUs, which causes that the key value contains '.module'. You should rename the provided pretrained model's keys."

could you tell me how to modify thoes keys in details , thank you : )

homurajiang commented 5 years ago

I just run the code to train CSR based on pretrained GCC model, which works well. I think you may use single GPU to train the model. However, our provided model is trained using multi GPUs, which causes that the key value contains '.module'. You should rename the provided pretrained model's keys.

I see your anwser of this

"I just run the code to train CSR based on pretrained GCC model, which works well. I think you may use single GPU to train the model. However, our provided model is trained using multi GPUs, which causes that the key value contains '.module'. You should rename the provided pretrained model's keys."

could you tell me how to modify thoes keys in details , thank you : )

gjy3035 commented 5 years ago

@homurajiang See also in https://github.com/gjy3035/C-3-Framework/issues/22#issuecomment-494287652