search

longcw / MOTDT

Real-time Multiple People Tracking with Deeply Learned Candidate Selection and Person Re-identification
MIT License
531 stars 130 forks source link

Have you fine-tune the Reid model on MOT dataset? #8

Open Kewenjing1020 opened 5 years ago

Kewenjing1020 commented 5 years ago

Hi @longcw , I've replaced the Reid part of extracting features by my own trained model, the evaluation result gets worse. Have your fine-tune the Reid model on the MOT dataset?

longcw commented 5 years ago

No, the reid model was trained on a combination of market1501 and duke reid dataset. I implemented the part-aligned reid (https://arxiv.org/abs/1707.07256) and got about 79.7 rank-1 on market 1501 dataset. Then the trained model was used for tracking without finetuning.

Kewenjing1020 commented 5 years ago

I found that the order of magnitudes of our Reid feature are different, maybe I need to normalize the Reid feature

Kewenjing1020 commented 5 years ago

Hi @longcw , after I normalize my Reid feature, I found the distribution of dists changes from approximately [0.1, 1.5] to [0.003, 0.08] (my feature size is 1x2048, and the original is 1x512). I think I only need to modify the min_ap_dist value and I actually tried some(0.05,0.06, etc.), but the result's always worse than the original ones. It's strange because I didn't change any other part and my Reid model is tested to be stronger on Market1501(90.0% rank-1), do you have any reminder on this point?

longcw commented 5 years ago

I am wondering why the distance changed to [0.003, 0.08] after you normalized the feature. L2 distance on two normalized features is same to the cosine distance (only different from some constants). So it should be in [0, 1] or something similar.

Second, I also tested resnet50 model with about 90% rank-1 on market1501 while the tracking performance is similar or even worse. I tried to train the model on market1501 and test it on duke and only got about 20%-30% rank-1. Reid models are very easy to overfit on the training set. So I train the model on three datasets to achieve better generation ability. Worse reid performance on the training set doesn't imply a worse tracking result. I think this is one reason you got a worse result.

To improve the model for tracking, the first thing is to improve the generation ability rather than achieve a higher score on the training set, unless you train the model on the tracking dataset directly.

Kewenjing1020 commented 5 years ago

Thank you, you've got a great point. I think the distance range still in [0,1], but the numbers are really small. I gonna train the Reid model on more dataset and let you know the results~

longcw commented 5 years ago

market1501, duke, and cuhk03.

But I am still wondering why you got the range of [0, 0.2]. It's an unnormal small range for cosine distance or L2 distance on normalized features.

For example, the following image is the distance distribution on MOT16-02. image

Kewenjing1020 commented 5 years ago

I didn't normalize the feature in training, that's the only reason I can think about...

Kewenjing1020 commented 5 years ago

I change the value of the min_ap_dist and visualize the result on MOT17, it seems not bad...

Kewenjing1020 commented 5 years ago

Hi, I tried with a reid model using normalized feature in training, the distance value is still small. While, as I add more data into training set, the distance value gets larger, e.g. [0,0.1] for model on Market1501, [0, 0.2] for model on Market1501+Duke, [0, 0.3] for model on Market1501+Duke+Msmt17. Maybe it's because of the network structure? For my model, I use resnet50 as backbone and get the conv5_x feature for further classification, while your model split the feature and align the feature with body part, so your feature map differs more in each part?

longcw commented 5 years ago

Maybe you can try to plot the distribution of positive and negative distances as I showed before, to find the best threshold and check the range.

ygren commented 5 years ago

So I have a question，how to train the reid model for MOT dataset or others? fine-tune the trained Reid model(we have trained on Market1501,Duke etc)on the MOT dataset or train on MOT dataset from other imgnet pretrain model? @longcw

Kewenjing1020 commented 5 years ago

@longcw I plot the cosine distance of MOT-02 with my Reid model: mot-02-dist-hist why do you have three different histograms in your image?

Kewenjing1020 commented 5 years ago

@ I'm trying to reproduce your reid model and I notice you define 'SpatialCrossMapLRN' in googleNet, what is this used for?

longcw commented 5 years ago

@Kewenjing1020 It's a LRN layer https://www.reddit.com/r/MachineLearning/comments/52jjmi/lrn_layer_with_googlenet/

longcw commented 5 years ago

@Kewenjing1020 @YgRen The reid model was trained by my pytorch re-implementation of https://github.com/zlmzju/part_reid.

Maybe I can share the training code later if you are interested in it.

ygren commented 5 years ago

@longcw Your work is very interesting！I'm very interested in your training process.Looking forward to your sharing so that I can reproduce your results.Thx！

shibu38 commented 5 years ago

Hi @longcw, @Kewenjing1020 can you tell me how to re-train Patchclassifier from scratch?

Kewenjing1020 commented 5 years ago

@shibu38 Hi, I didn't try to reproduce patch classifier yet.

Kewenjing1020 commented 5 years ago

@longcw I tried to retrained reid by adding a classifier after your network and fine-tune from your model, but the code always corrupts. Then I removed the LRN layer, then it runs well but with a really bad result...

shibu38 commented 5 years ago

@Kewenjing1020 Hi, do you have any idea how to do that?? or Do you have any plan in future to that??

Kewenjing1020 commented 5 years ago

@longcw Hi, my features distance finally become normal. It's because the image preprocess part doesn't consist with that in my model training code. Shame of me taking so long to find the bug. TT

Crazy-stones commented 5 years ago

Is googleNet for Reid? And squeezeNet for what?

Crazy-stones commented 5 years ago

Are they both from https://github.com/zlmzju/part_reid?

longcw commented 5 years ago

Please read the paper.

Crazy-stones notifications@github.com 于2019年5月10日周五 下午5:22写道：

Are they both from https://github.com/zlmzju/part_reid?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/longcw/MOTDT/issues/8#issuecomment-491222301, or mute the thread https://github.com/notifications/unsubscribe-auth/ABPJJAARU73XSA4OFUMYQFDPUU5D7ANCNFSM4F3PKZYA .

Crazy-stones commented 5 years ago

I generally know that you get the re-id model from https://github.com/zlmzju/part_reid which uses part-aligned reid and GoogleNet. But I still do not understand why you need two models and what are their functions?

Crazy-stones commented 5 years ago

@longcw Can you solve my question? Thanks!

paganpasta commented 5 years ago

I generally know that you get the re-id model from https://github.com/zlmzju/part_reid which uses part-aligned reid and GoogleNet. But I still do not understand why you need two models and what are their functions?

Section 3.2, 3.4 here @Crazy-stones: As the author mentioned, reading the paper might help.