ONNX export may fail for Hiera due to `math.sqrt` and python type `int` casts

[xenova]

Similar to a previous issue (https://github.com/huggingface/transformers/pull/31311) we had with vision models. cc @merveenoyan @amyeroberts @NielsRogge

Offending lines:

https://github.com/huggingface/transformers/blob/5c1027bf09717f664b579e01cbb8ec3ef5aeb140/src/transformers/models/hiera/modeling_hiera.py#L340

https://github.com/huggingface/transformers/blob/5c1027bf09717f664b579e01cbb8ec3ef5aeb140/src/transformers/models/hiera/modeling_hiera.py#L344

We should probably abstract interpolate_pos_encoding, since this is reused across many different vision architectures.

[qubvel]

Hi @xenova, there are more issues with the current implementation of interpolate_pos_encoding

• https://github.com/huggingface/transformers/issues/32410#issuecomment-2293359788

Maybe it's better to rewrite interpolation using size argument instead of scale to avoid errors described in the issue above, will it be compatible with ONNX export? WDYT?

[xenova]

Maybe it's better to rewrite interpolation using size argument instead of scale to avoid errors described in the issue above, will it be compatible with ONNX export? WDYT?

Yes I agree! Some other issues/PRs where this is discussed:

• https://github.com/YvanYin/Metric3D/issues/117#issue-2364404569
• https://github.com/huggingface/optimum/pull/1580
• https://github.com/huggingface/transformers/pull/30065

I'll try draft a PR for this soon.