n_output_channels

[liushawn618]

Hello,

Congrats on the great work again. What's the n_output_channels and downsample_rate of hiera_base_224?

[dbolya]

B has 768 output channels with a 32x total downsample rate (224 -> 7).

[liushawn618]

Thank you for your timely reply. After changing n_output_channels into 768, I get RuntimeError: mat1 and mat2 shapes cannot be multiplied 64x1000 and 768x512. And then I just change it like this Now it's running, but is it ok to do like this?

[dbolya]

Can you share which script you're trying to use this with? Whether that's correct depends on what those variables specifically mean in that script.

[liushawn618]

I use this script "python scripts_method/train.py --demo --backbone hiera --setup p1 --method arctic_sf --trainsplit train --valsplit minival"

[dbolya]

I use this script "python scripts_method/train.py --demo --backbone hiera --setup p1 --method arctic_sf --trainsplit train --valsplit minival"

In which repository is this?

[liushawn618]

I want to use hiera to extract features in this repository: https://github.com/zc-alexfan/arctic

[liushawn618]

The result seems not very good

[liushawn618]

The model is this https://github.com/zc-alexfan/arctic/blob/master/src/models/arctic_sf/model.py

And I changed 3 lines

[dbolya]

Can you push your changes to a local repository? From what I can tell, 768 is the correct number of final features, so something may be going wrong elsewhere.

[liushawn618]

Thank you for your kind response. My local respository is https://github.com/liushawn618/hiera_arctic_sf

[liushawn618]

The yellow line shows hiera as backbone. It seems to perform well during training but not so good during validation.

[dbolya]

Ah, I see what's going on. You're running Hiera in classification mode, not feature extraction. The backbone is returning an imagenet classification (batch x n_classes, which is why you see 64x1000).

The correct parameters are n_output_channels of 768, downsample_rate of 4 (though this seems to be unused) and to revert the changes to the second argument to head_r, head_l, head_o.

Then to get intermediate features for Heira, replace the backbone evaluation with:

if self.args.heira:
    _, features = self.backbone(images, return_intermediates=True)
    features = features[-1]  # Get features from last stage
else:
   features = self.backbone(images)

[liushawn618]

Thank you so much! I try again with your help and now it's on the right track!

[dbolya]

Great! Feel free to reopen this issue if you need more help.