Some concerns about the code

[22TonyFStark]

Hello, I have some concerns about the following lines:

1. In this line, the order of image features is reversed, according to the image_encoder.py, thus the final feature list is [layer last, layer 9, layer 6, layer 3]: https://github.com/med-air/3DSAM-adapter/blob/6ba7e037047df46b6a8d99eb81c82149e43f8c53/3DSAM-adapter/train.py#L172 However, in this line, the point prompt encoder is used if i == 3, which means the prompt encoder uses the feature from the 3rd layer, and this may be different from Fig.1 in the paper, where the prompt encoder uses the last layer. https://github.com/med-air/3DSAM-adapter/blob/6ba7e037047df46b6a8d99eb81c82149e43f8c53/3DSAM-adapter/train.py#L197 In my humble opinion, the above line should be if i == 1 instead.

2. from this line, the inputs of the prompt encoder include image feature, points, and size: https://github.com/med-air/3DSAM-adapter/blob/6ba7e037047df46b6a8d99eb81c82149e43f8c53/3DSAM-adapter/train.py#L199 However, according to this line, the 3rd input of the prompt encoder is the mask, which is not used, and the 4th input is the size. https://github.com/med-air/3DSAM-adapter/blob/6ba7e037047df46b6a8d99eb81c82149e43f8c53/3DSAM-adapter/modeling/Med_SAM/prompt_encoder.py#L312 This may lead to the wrong points' position, and the latter grid sample will get the wrong image features: https://github.com/med-air/3DSAM-adapter/blob/6ba7e037047df46b6a8d99eb81c82149e43f8c53/3DSAM-adapter/modeling/Med_SAM/prompt_encoder.py#L124 I think deleting the unused mask input of the prompt encoder may help: # masks = None,

[peterant330]

Hi,

Thanks for pointing this out. You are right about these two issues. We did make some mistakes when merging the code. We will soon fix the problem.

[22TonyFStark]

Thanks for your explanation~