Open lsorber opened 6 days ago
Hello!
I understand this feature request, but I'm afraid it's likely not going to be feasible. To explain why, let me first explain how Sentence Transformer models are structured.
A Sentence Transformer model is a torch.nn.Sequential
class containing multiple torch.nn.Module
subclasses, usually:
And perhaps some more. It is also possible to add custom modules (docs) by subclassing torch.nn.Module
and implementing some functions.
The ONNX/OpenVINO support was implemented inside of the Transformer
class, with either AutoModel
, ORTModelForFeatureExtraction
or OVModelForFeatureExtraction
being used depending on the backend
.
So, there's a few reasons that I can't remove the torch
dependency:
torch
is still used for all other modules like Pooling.torch
dependency would prevent me from supporting custom modules like this one.torch
dependency would prevent me from being able to "export on the fly" like is currently possible. In short, the model doesn't need to already be exported.torch
as well: https://github.com/huggingface/optimum/blob/main/setup.py#L19In short, I'm afraid I can't get around it.
Understood, thank you for looking into it!
After seeing this tweet [1] by @tomaarsen in which he announced a set of new ONNX and OpenVINO models, I was hoping to run those models without having to depend on torch. Would it be possible to make torch an optional dependency?
[1] https://x.com/tomaarsen/status/1846124176085590514