Feature request: make torch an optional dependency when using other backends

[lsorber]

After seeing this tweet [1] by @tomaarsen in which he announced a set of new ONNX and OpenVINO models, I was hoping to run those models without having to depend on torch. Would it be possible to make torch an optional dependency?

[1] https://x.com/tomaarsen/status/1846124176085590514

[tomaarsen]

Hello!

I understand this feature request, but I'm afraid it's likely not going to be feasible. To explain why, let me first explain how Sentence Transformer models are structured.

A Sentence Transformer model is a torch.nn.Sequential class containing multiple torch.nn.Module subclasses, usually:

1. Transformer
2. Pooling

And perhaps some more. It is also possible to add custom modules (docs) by subclassing torch.nn.Module and implementing some functions.

The ONNX/OpenVINO support was implemented inside of the Transformer class, with either AutoModel, ORTModelForFeatureExtraction or OVModelForFeatureExtraction being used depending on the backend.

So, there's a few reasons that I can't remove the torch dependency:

1. ONNX/OV is only used for to replace the underlying Transformers-based model - torch is still used for all other modules like Pooling.
2. Removing the torch dependency would prevent me from supporting custom modules like this one.
3. Removing the torch dependency would prevent me from being able to "export on the fly" like is currently possible. In short, the model doesn't need to already be exported.
4. Optimum, the project that I rely on heavily for the ONNX/OV backends, requires torch as well: https://github.com/huggingface/optimum/blob/main/setup.py#L19

In short, I'm afraid I can't get around it.

• Tom Aarsen

[lsorber]

Understood, thank you for looking into it!