search

cvcode18 / imbalanced_learning

102 stars 20 forks source link

Test setting #8

Closed huilongan closed 5 years ago

huilongan commented 5 years ago

Hi,

I noticed the test code is exactly the same as training part. The BatchNorm operation should be different in these two sessions. Have you handled that in the code? I did not find it.

Huilong

nsarafianos commented 5 years ago

Hi @huilongan

Unlike PyTorch that requires handling this case as you're suggesting (link from Soumith), in MXNet you don't need to worry about that as it's handled internally (I think it's taken care of from the autograd.record that is done at training). This behavior is similar with Dropout in which you can still have a probability > 0 at test time and you will see that it's actually not used and that the results are the same with the prob=0.

Let me know if that answers your question.

huilongan commented 5 years ago

Thanks for your response. I will check that. I am new to MXNet, so sometimes I got confused. Thank you very much. Btw, I tried to use another dataset named PA-100K with 26 attributes, and the result is not so good, Map is like about 0.63. Could you give me some advice on that? thanks.

nsarafianos commented 5 years ago

Hi @huilongan

I just checked the original paper and especially Table 2 with the quantitative results. 1) I would first ensure that you resize all pedestrians to 128x256 (WxH) and I would fine-tune a ResNet-101 w/o anything else w/ SGD learning rate schedulers, augmentation etc. Spend some time here since this should get you to a good performance.
2) I would look at the class imbalance for each attribute to have a better idea. Find the class imbalance on the training set and use these weights to do cost-sensitive learning (weighted BCE). Does it improve? 3) Build a simple attention mechanism. This can be what we do in this work or others (harmonious attention is my favorite and they provide code). Plug it at the end of Block 3 of ResNet. Check if it improves 4) Try the focal loss/weighted focal loss and check how it improves upon the baseline

huilongan commented 5 years ago

Thanks for your quick response. I will do it today and let you know the result.