inaturalist / iNaturalistMLWork

0 stars 0 forks source link

in-depth comparison of the image preprocessing pipeline #24

Open alexshepard opened 4 months ago

alexshepard commented 4 months ago

this came up as followup todos during the bounding box experiments.

across the clades, evaluate the following different approaches:

also, visualize the different approaches in a slide or presentation or notebook so that people know what we're talking about.

alexshepard commented 4 months ago

I made a comparison between three different preprocessing strategies:

  1. no crop, just squeeze
  2. central crop, then squeeze
  3. resize with crop, no squeezing
alexshepard commented 4 months ago

here is how the "no crop, just squeeze" approach transforms an image

Image

alexshepard commented 4 months ago

here is how the "central crop, then squeeze" approach transforms an image

Image

note this is how we do things in production now

alexshepard commented 4 months ago

here is how the "resize with crop, no squeezing" approach transforms an image

Image

alexshepard commented 4 months ago

Image

here's a look at performance metrics across our normal categories for each approach.

based on this, I'm recommending we switch to using tf.image.resize_with_crop_or_pad to preprocess our images. I'll make a pull request for this in the inatVisionAPI repo.

alexshepard commented 4 months ago

I made a PR for this here https://github.com/inaturalist/inatVisionAPI/pull/17 and requested that Patrick take a look