Pretrained weights.

[Jmipar-k]

Hello,

I have seen your great work on surgical DINO and endoDAC!

Could you make the pretrained weights for surgical DINO public?

Also, I would appreciate it if you could add the instructions for training, or finetuning surgical DINO.

I am trying to use your work for my own research on medical fields.

It would be thankful if you could share the pretrained weights with us.

Thank you so much!

[BeileiCui]

Hi, I did not put the supervised training code for Surgical-DINO because the code was not so well written so I removed them from my server. The training code is just a supervised training process optimizing the ground truth depth and the estimated depth.

If you are looking for Self-Supervised fine-tuning methods, I have uploaded more comprehensive codes including evaluation, training and visualization in the EndoDAC repo. You may also find a pretrained checkpoint there.

[Jmipar-k]

I appreciate your fast response.

I get what you mean, so I will check out your new repository (endoDAC)

I have a question.

I am trying to find a foundation model to replace the CLIP-T model in the image a shared with you. It is because I am trying to solve a medical task in endoscopy field. Do you think replacing that model with the endoDAC model is possible? I do not need the depth informations but I need a foundation model to encode endoscopy images into some kind of feature or vectors. I was thinking that I could get the feature(or vector) right before it gets decoded into depth images(like the framework in surgical DINO) and put it in the Unet part(in the shared image). When I saw the framework of endoDAC, it looked pretty different from surgical DINO's framework so i was worried if I would be able to use endoDAC instead of surgical DINO.

Sorry for the long question, I am a beginner in medical field, especially specific fields like endoscopy. Thank You So Much!

[Jmipar-k]

I have a additional question,

Does the fine-tuned foundation models include information of small intestine endoscopy images?

I was wondering if the pretrained weight would be able to cover my custom small intestine endoscopy dataset.

Thank you again!!

[BeileiCui]

Hi @Jmipar-k Sorry for my late response. SurgicalDINO and EndoDAC are both fine-tuning methods, so I would not recommend you to replace CLIP-T model with them. I suggest you try a surgical foundation model (which is not so many I think), for example medSAM (I don't remember the exact name, it was published on Nature Communication recently)

For the next question, I think SCARED does not includes small intestine endoscopy so we do not have them sorry, you could fine-tune on some small intestine endoscopy datasets if you like.

[Jmipar-k]

Thank you so much for the kind comment.