elixir-nx / bumblebee

Pre-trained Neural Network models in Axon (+ 🤗 Models integration)
Apache License 2.0
1.26k stars 90 forks source link

Change image size to maps in image featurizers #329

Closed jonatanklosko closed 4 months ago

jonatanklosko commented 4 months ago

Initially CLIP was using a single integer as size and always resized the shortest edge. Now it supports both fixed height-width and shortest edge resizing (there is padding on top of it, so the shape is still static).

I updated all featurizers to use maps, such as %{height: 224, width: 224} and %{shortest_edge: 224}. This matches hf/transformers and in the feature we may need to support more of these (for other featurizers).