ZF4444 / MMAL-Net

This is a PyTorch implementation of the paper "Multi-branch and Multi-scale Attention Learning for Fine-Grained Visual Categorization (MMAL-Net)" (Fan Zhang, Meng Li, Guisheng Zhai, Yizhao Liu).
249 stars 57 forks source link

There is something wrong #9

Closed b21727795 closed 4 years ago

b21727795 commented 4 years ago

Hi! First of all, congratulations on your work. I want to use your model as a pretrained model for my own work.In a dataset containing bird images, I would like to use it to eliminate background or images without birds. When I run test.py on my dataset, acc comes out very low (1%).The following error message appears.("there is one img no intersection")What is the main problem.

And also I want to ask that,In your paper you said that this model extracts object location(bounding box) and discriminative parts. But when I investigate your code,It seem you are using bounding box information but you are not using directly or indirectly.I cant understand how your code works.And how can be apply your model for my custom dataset for related purpose I said before.

Thanks and be healty,Harun Alperen

ZF4444 commented 4 years ago

Hi! Thanks! 1.acc comes out very low (1%): Whether your dataset class is consistent with CUB's class? 2."there is one img no intersection": This mean that the mask from conv_5c don't have intersection with the mask from conv_5b, and the object image is the raw image in this situation. 3.About using bounding box, I use it in https://github.com/ZF1044404254/MMAL-Net/blob/199c1706695ba0f645809d3568881816fbe2fb70/networks/model.py#L105 4.About your purpose of eliminating the background, you can use the predicted bounding box like the third answer. 5.About your purpose of eliminating images without birds, maybe you should use a object detection method.

b21727795 commented 4 years ago

Thank you for quick answer.

I wonder something. When I run the test.py according to CUB dataset .It yields object branch accuracy .Why It returns object branch accuracy.Total acc coming from 3 branches,isn't it.Also for custom dataset. Should I add new code for dataset.py.I'm asking because my bird dataset, actually it is a CUB dataset but source is different.So,I have 200 class and for this class I simply extract images.txt class_label.txt train_test split.txt .But I have no information about bounding boxes so for custom dataset(basically it is CUB dataset )should I add bounding_boxes.txt. I assume that your implementation doesn't require bounding box information.

Thanks,Harun Alperen

b21727795 commented 4 years ago

If I need bounding box information for custom dataset, Will I extract this information via training the network.

ZF4444 commented 4 years ago
  1. After our repeated experiments, the best classification performance is simply obtained by the result of object branch.
  2. It's no need to add bounding_boxes.txt, which is used to verify the object localization performance of AOLM. But you should make sure that your images.txt class_label.txt train_test split.txt is same with official. Besides, you should edit dataset.py because of the lack of bounding_boxes.txt.
  3. Yes, you can get bounding box information predicted by the AOLM, but please note that the predicted bounding box information are not absolutely right.
b21727795 commented 4 years ago

Thank you

bruceykh commented 2 years ago

@b21727795 could you tell me how to solve the problem "there is one img no intersection".Great appreaciaton!