search

huggingface / exporters

Export Hugging Face models to Core ML and TensorFlow Lite
Apache License 2.0
622 stars 46 forks source link

Experimental: do not annotate output shapes #10

Closed pcuenca closed 1 year ago

pcuenca commented 1 year ago

If I'm reading this comment right, it looks like annotating flexible output shapes is going to be very hard to support for ML Program conversions. Furthermore, annotating non-flexible output shapes still doesn't work in the presence of flexible input shapes, as the conversion carries symbolic information about the shapes through the network. Letting output shapes float seems to work, at least for bert: https://huggingface.co/pcuenq/gists/blob/main/flexible_inputs_only.ipynb.

I'm not sure about the impact of this change. If we decide to go this way, we might want to keep get_flexible_outputs() for information and testing purposes. We could create a test (if it doesn't already exist, I didn't check) that runs different sequence lengths and ensures output lengths are consistent with inputs.

hollance commented 1 year ago

Interesting. If they're saying this is the wrong technique to use, we shouldn't be using it. But it wouldn't surprise me if removing this breaks other stuff. ;-)