search

mhamilton723 / STEGO

Unsupervised Semantic Segmentation by Distilling Feature Correspondences
MIT License
712 stars 143 forks source link

Potsdam IR channel #25

Closed amjams closed 1 year ago

amjams commented 2 years ago

Hello!

In src/data.py there's the following comment "# TODO add ir channel back". I'm guessing that dataset was trained without the IR channel?

I ask because I'm interested in training on a custom dataset with more than RGB channels, somewhat akin to Potsdam's RGBIR. What would you recommend for doing that? Would it be as simple as creating a copy of the DirectoryDataset class and modifying it? Or would it require more elaborate changes?

Thanks, AJ

mhamilton723 commented 2 years ago

Great question @amjams, i didnt add this because DINO pretrained on ImageNet has only 3 channels. You might however be able to add a channel to DINO and fine-tune on potsdam prior to using in STEGO though. If you do please feel free to share your model and results and we will give your work a call-out :)