search

huggingface / pytorch-image-models

The largest collection of PyTorch image encoders / backbones. Including train, eval, inference, export scripts, and pretrained weights -- ResNet, ResNeXT, EfficientNet, NFNet, Vision Transformer (ViT), MobileNetV4, MobileNet-V3 & V2, RegNet, DPN, CSPNet, Swin Transformer, MaxViT, CoAtNet, ConvNeXt, and more
https://huggingface.co/docs/timm
Apache License 2.0
31.38k stars 4.7k forks source link

[FEATURE] Multi-Label Classification #452

Open liangshi036 opened 3 years ago

liangshi036 commented 3 years ago

How do I apply this nice repo to Multi-Label Classification task ? That is , a image has multi labels，I'd like to get the each label's confidence.

sound like add some features in dataset.py? or more than this file?

Thanks.

rwightman commented 3 years ago

@liangshi036 I do have an (eventual) plan to add support for BCE and/or MultiLabelSoft margin losses for use in multi-label tasks and include support for a suitable dataset. Have a number of other significant refactorings of the train code to support TPU ahead of that in the queue though....

liangshi036 commented 3 years ago

That's Awesome. Looking Forward to your update!

thanks for this great job which keep us in touch with SOTA things.

ravich3373 commented 3 years ago

looking forward to this.

Thank you.

yang-ruixin commented 3 years ago

@rwightman Thank Ross for his great work. I've implemented multi-label classification based on his code. Here is the link: https://github.com/yang-ruixin/PyTorch-Image-Models-Multi-Label-Classification

Cheers, Ruixin

liangshi036 commented 3 years ago

@yang-ruixin , seems you're not using BCE losses. have you test your code with multi-label classification task ?

yang-ruixin commented 3 years ago

@yang-ruixin , seems you're not using BCE losses. have you test your code with multi-label classification task ?

Hi there,

Sure I've tested my code, with fashion-product-images dataset and another private dataset. BCE loss, B stands for binary, but in our case, each label has more than two classes, hence you cannot use BCE loss. The loss I used is the same as Ross' loss. In MultiLabelModel class, get_loss method, the input argument loss_fn is defined in lines 567~578 in ./train.py.

Ruixin

liangshi036 commented 3 years ago

BCE (nn.BCEWithLogitsLoss) is a elegant way to treats output independently (as logit loss) in multi-label classification task . maybe the same as you do in your repo.

you can post your test result by the way.

abhigoku10 commented 2 years ago

@yang-ruixin @liangshi036 @rwightman thanks for sharing your source code, i am trying to do attribute recognition for the person using transformers which is a multilabel classification problem statement, can you please share your thought on the following points

  1. I am using coat,pit architectures can remove the classifier head and add my classifier? can you share some references on how to do it
  2. is bce loss better than MultiLabelSoft loss?
  3. can we apply Spatial transformers insider transformers for localization of attributes detected

Thanks in advance