search

SHI-Labs / Neighborhood-Attention-Transformer

Neighborhood Attention Transformer, arxiv 2022 / CVPR 2023. Dilated Neighborhood Attention Transformer, arxiv 2022
MIT License
1.05k stars 86 forks source link

ONNX #74

Closed idanpd closed 1 year ago

idanpd commented 1 year ago

Hi, trying to convert to onnx but natten module can not be converted. Is there any solution for this kind of problem ?

issue created when getting to:

_C.natten2dav_forward( attn, value, dilation)

alihassanijr commented 1 year ago

Hello and thank you for your interest.

Unfortunately that is not surprising, because Neighborhood Attention is a "custom" torch operation, and therefore doesn't support conversion to ONNX. In theory we --could-- try and avoid the error, but then the problem would be that you'd still need to run that ONNX model on a PyTorch environment only, which kind of defeats the purpose.

I would however recommend raising this issue in NATTEN, because that's where we've moved our entire extension, and therefore this repository only uses that package. As long as NATTEN starts supporting ONNX, the issue would be resolved for models found in this repository. However, even if it does, it'll still have to be implemented for your target framework in order to run there.

I hope this answers your question.

alihassanijr commented 1 year ago

Closing this issue since it exists in NATTEN: https://github.com/SHI-Labs/NATTEN/issues/17 .