I am currently working on deploying the Seamless M4T model for text-to-text translation on a Triton server. I have successfully exported the text.encoder to ONNX and traced it using torch.jit.trace. However, I am encountering issues when trying to export the text.decoder to ONNX or trace it using torch.jit.trace. The process is being blocked by IncrementalStateBag.
Steps to Reproduce
Exporting text.encoder:
Successfully exported to ONNX.
Successfully traced using torch.jit.trace.
Exporting text.decoder:
Attempted to export to ONNX.
Attempted to trace using torch.jit.trace.
Encountered an issue with IncrementalStateBag.
Observed Behavior
When attempting to export or trace the text.decoder, the process is interrupted by IncrementalStateBag, preventing successful completion.
Expected Behavior
The text.decoder should be successfully exported to ONNX or traced using torch.jit.trace, similar to the text.encoder, enabling deployment on the Triton server for text-to-text translation.
Description
I am currently working on deploying the Seamless M4T model for text-to-text translation on a Triton server. I have successfully exported the
text.encoder
to ONNX and traced it usingtorch.jit.trace
. However, I am encountering issues when trying to export thetext.decoder
to ONNX or trace it usingtorch.jit.trace
. The process is being blocked byIncrementalStateBag
.Steps to Reproduce
Exporting
text.encoder
:torch.jit.trace
.Exporting
text.decoder
:torch.jit.trace
.IncrementalStateBag
.Observed Behavior
text.decoder
, the process is interrupted byIncrementalStateBag
, preventing successful completion.Expected Behavior
text.decoder
should be successfully exported to ONNX or traced usingtorch.jit.trace
, similar to thetext.encoder
, enabling deployment on the Triton server for text-to-text translation.Environment
torch.jit.trace
Questions and Assistance Needed
IncrementalStateBag
when exporting or tracing thetext.decoder
?text.decoder
?Any assistance or guidance on this matter would be greatly appreciated. Thank you!
Additional Information
Please let me know if there is any additional information required to diagnose the issue.
Thank you for your support!