I'd like to use this feature extractor for standard ViT (CLIP) model. I found that the naive output of attention layer is tuple shape (activation, None). Since this feature extractor is also used in icnn.py, it will raise error when we perform reconstruction analysis using attention layer. One way to avoid this issue is just selecting the first element when the output is a tuple.
if type(output) is tuple:
features[layer] = output[0]
else:
features[layer] = output
https://github.com/KamitaniLab/bdpy/blob/ec5afa9ce818b667fc85678a75861c84d15c0f27/bdpy/dl/torch/torch.py#L85
I'd like to use this feature extractor for standard ViT (CLIP) model. I found that the naive output of attention layer is tuple shape (activation, None). Since this feature extractor is also used in icnn.py, it will raise error when we perform reconstruction analysis using attention layer. One way to avoid this issue is just selecting the first element when the output is a tuple.