search

microsoft / arcticseals

A deep learning project in cooperation with the NOAA Marine Mammal Lab to detect & classify arctic seals in aerial imagery to understand how they’re adapting to a changing world.
MIT License
33 stars 14 forks source link

IR normalization with current variable px distributions #32

Open readicculus opened 5 years ago

readicculus commented 5 years ago

Not a collaborator so could not re-open the closed normalization issue but I've been working on this on my own and have spent a lot of time on the IR imagery. The current issue, which I think its not possible to get around, is that there is no standard distribution of values nor can it be expected. This means that it's not too difficult to do percentile normalization on the given dataset but when it comes to images without hotspots the normalization causes warmer areas to look like hotspots.

There are a lot of pools of water that look very similar to seals in shape and size but are not. Without some sort of temperature to pixel correlation I've not found a way to do this so that it won't detect a ton of FPs on images without seals or other animals.

It sounds like in future flights the camera will include a temp correlation to at least the max pixel value but until then I'm thinking that the IR data is unfortunately relatively useless. IR imagery is suuuper important as I was getting an F1 score of around .93 on the labeled images and its WAYYYY faster to localize on them than the RGB imagery. Ideally we need to achieve about 3fps localization to be able to run this live on the plane which is seeming impossible with the large rgb images where it's taking me about 1-4 secs per image depending on the model and chip size.

Any feedback would be great as getting IR to work would be a massive step forward in this project.

readicculus commented 5 years ago

Also just to introduce myself I'm Yuval, CS student at UW and working on this as an internship for about 6 months now.

jomalsan commented 5 years ago

Hi Yuval, when working with IR imagery, you need to resist normalizing each image independently, because like you said it will eliminate all useful information from the image. The approach that we took was to do a global normalization on a per camera basis. This normalization was based on adjusting the histogram of all measured values for each camera to be approximately the same and was the best that we could do with the current state of the data. Documentation should be in the ir-normalization folder on how this was done.

In future flights having a properly calibrated camera will be the best path forward and eliminate all of this hand-wavy calibration work. Until then, make sure that all computations you do on IR images are applied to all of them to avoid washing out the valuable information