search

leeyeehoo / CSRNet-pytorch

CSRNet: Dilated Convolutional Neural Networks for Understanding the Highly Congested Scenes
642 stars 259 forks source link

MAE for WorldExpo, cannot reach 8.6 #28

Closed LaiPiXiong closed 5 years ago

LaiPiXiong commented 5 years ago

HI, I fine-tuning on WorldExpo dataset, but it cannot reach 8.6 as paper said. I only got MAE 25.494. Can you give me some advices? @leeyeehoo

LaiPiXiong commented 5 years ago

I got results as papers said.

liwenxi commented 5 years ago

Can you tell what did you do on training WorldExpo and Mall dataset? @LaiPiXiong

SherlockHua1995 commented 5 years ago

Hello， LaiPiXiong. I am trying to train the model on the WorldExpo dataset, but I am not sure about how to deal with with this dataset. Would you plz release your code and I can refer your pytorch code for my own script(with keras).

yxxxxxxxx commented 5 years ago

@LaiPiXiong Hello, I tried the same setting on WorldExpo in the paper but only got MAE at around 11, could you please show your training details? I'd appreciate it if you can help me. Thank you very much!

xxxwuwq commented 4 years ago

I'm confused by the test method on different scene of WorldExpo'10. And i have tried two test method. (training using CSRNet, get a lot of model from different epoch)

The first one, choose a best model, and then test on different scene, (for one model, get five scene mae result, and calculate the average mae), this method can't get the result mae=8.6, only get the result about 12;

The second, for each scene, choose one model to test, (it means that, maybe choose five different models for different scene to get five best results), and calculate the average mae result. This method could get the result that close to mae=8.6.

Is Right or not that using the second method?

liwenxi commented 4 years ago

@wwq-online Both methods are introduced in [1], and the data referenced in CSRNet belongs to the second method. However, as far as I know, some other models better than CSRNet use the first method. It is recommended not to stick to this problem, but try to find some problems, and find a more general algorithm.

In addition, some new video datasets have been proposed. You can evaluate your model on them.

[1] Zhang C, Li H, Wang X, et al. Cross-scene crowd counting via deep convolutional neural networks[C]// In CVPR 2015

hellowangqian commented 4 years ago

@wwq-online The first one looks reasonable to me unless you can find a smart way to select the optimal model for each of the test scenes without using the test labels (as the original Expo dataset paper did). I've tested a few models using the first one, and got similar results as yours (MAE=11~12).