search

med-air / Endo-FM

[MICCAI'23] Foundation Model for Endoscopy Video Analysis via Large-scale Self-supervised Pre-train
Apache License 2.0
154 stars 15 forks source link

Question for pretrain pth #14

Open November198 opened 6 months ago

November198 commented 6 months ago

Can you provide the weights for all the comparative methods trained on your dataset? Thank you~

Kyfafyd commented 6 months ago

Hi, @Dorothy-h357h Thanks for your interest in our work! Unfortunately, these weights have been deleted before due to storage issue. You can directly train these models on our dataset with their release codes. The only works are adapting the backbone ViT/B-16 and the dataloader.

Feel free to comment here if you meet any problem during reproduction.

November198 commented 6 months ago

Hi, @Dorothy-h357h Thanks for your interest in our work! Unfortunately, these weights have been deleted before due to storage issue. You can directly train these models on our dataset with their release codes. The only works are adapting the backbone ViT/B-16 and the dataloader.

Feel free to comment here if you meet any problem during reproduction.

Thanks for your reply. During my reproduction process, I found that ST-Adapter involves supervised training, whereas our pretraining dataset lacks labels. How is this addressed?

Kyfafyd commented 6 months ago

We use the ST-Adapter in ViT and just finetune ST-Adapter during pre-training, i.e., the other parts are fixed.

November198 commented 6 months ago

We use the ST-Adapter in ViT and just finetune ST-Adapter during pre-training, i.e., the other parts are fixed.

Can you provide a piece of code for pretraining ST-Adapter? Thank you

Kyfafyd commented 6 months ago

You may refer to this file of ST-Adapter: https://github.com/linziyi96/st-adapter/blob/main/models_adapter.py

Only the adapters in ViT are trained, just like: https://github.com/linziyi96/st-adapter/blob/b914b388f45ddd1968348ae2b1590a17eca75f8f/models_adapter.py#L196-L199 And video forwarding is like: https://github.com/linziyi96/st-adapter/blob/b914b388f45ddd1968348ae2b1590a17eca75f8f/models_adapter.py#L207-L230

November198 commented 6 months ago

I'm very sorry, but I still don't understand how you compare with methods like ST-Adapter and VCL. At the same time, I also found that ProViCo and CORP do not have open source code. Can you please explain in detail?