Is the named concepts used in the code?

[nashswift]

Dear author, Thank you and your research team for your contribution! I have a question. In your article, you mention "We use the extracted and named concepts as a concept bottleneck layer...", but in the code, I do not see the concepts obtained after the second step "automated concept naming" being used. Instead, the SAE trained in the first step is used. Where are these automatically named concepts used?

Thanks a lot！

[sukrutrao]

Hi @nashswift, the output of the SAE encoder constitutes the concept bottleneck, which is then used to form concept bottleneck models on downstream datasets. Independent of this, each of the concepts in this same concept bottleneck is assigned a name using the concept naming step. In particular, the automated concept naming step does not change the concepts in the SAE concept bottleneck.

[nashswift]

So as you said, the second step of automatic naming has no effect on the training of downstream tasks, which means that this step can be regarded as a kind of "visualization" of the concepts acquired by the SAE encoder, right?

[SwetaMahajan]

Hi @nashswift, the automated naming only takes the already learned dictionary vector and names it according to which text embedding is closest to it and as you rightly said, it does not have any effect on the training of the downstream tasks. The task-agnosticity plots (top activating images for the corresponding concept) you see in the paper are to visualise the interpretability and the consistency of the discovered concepts and the name assigned verifies whether naming is appropriate. The top activating images are independent of the naming as it depends only on the trained SAE.

[nashswift]

I think I understand what you mean. Thank you for your patience and once again thank you and your team for your research contributions！

[sukrutrao]

Thank you for your interest, and please feel free to let us know if you have any other questions.