search

yewsiang / ConceptBottleneck

Concept Bottleneck Models, ICML 2020
MIT License
174 stars 30 forks source link

Order of concepts in attribute_label #15

Closed JackFurby closed 2 years ago

JackFurby commented 2 years ago

Hello,

I am trying to reproduce the model but I am having an issue translating the concept vector to concept labels. For instance, with the classification class 9 (Brewer_Blackbird) the concept vector in the dataset is [0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0] but that translates to the following concepts when using the concepts commented here:

has_bill_shape::spatulate
has_wing_color::orange
has_upperparts_color::orange
has_underparts_color::orange
has_underparts_color::buff
has_back_color::orange
has_upper_tail_color::orange
has_breast_color::orange
has_throat_color::orange
has_eye_color::orange
has_bill_length::longer_than_head
has_forehead_color::orange
has_under_tail_color::orange
has_nape_color::orange
has_belly_color::orange
has_belly_color::buff
has_size::large_(16_-_32_in)
has_shape::perching-like
has_back_pattern::multi-colored
has_tail_pattern::multi-colored
has_primary_color::orange
has_leg_color::orange
has_bill_color::orange
has_crown_color::orange
has_crown_color::buff

It looks as if the concepts are off by one or in a different order to the order in the comment? Would it be possible to know if the concepts are correct and if the order is correct? If the concepts are not correct could the correct ones be listed?

I have tried to re-run dataset processing on my machine but this throws errors

thaonguyen19 commented 2 years ago

Hi, The list of 112 attributes indicated in https://github.com/yewsiang/ConceptBottleneck/blob/master/CUB/generate_new_data.py#L71 is not off by 1. Can I ask how you generated the concept vector [0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0] for Brewer_Blackbird?

JackFurby commented 2 years ago

Hi @thaonguyen19,

Thank you for your reply. I am getting the concept vector from the attribute_label in the dataset pickle files.

thaonguyen19 commented 2 years ago

I see where the problem is: note that attributes.txt is 1-indexed, while the list of 112 attributes we provided is 0-indexed. If you take this into account, the list of attributes for class 9 would be has_bill_shape::all-purpose, has_wing_color::black, etc ...

JackFurby commented 2 years ago

That is it. I had made the assumption both attribute.txt and the 112 attributes had the same indexing.

Thank you for helping solve this issue.