pytorch / text

Models, data loaders and abstractions for language processing, powered by PyTorch
https://pytorch.org/text
BSD 3-Clause "New" or "Revised" License
3.49k stars 815 forks source link

Torchscriptable T5 generation #2146

Closed joecummings closed 1 year ago

joecummings commented 1 year ago

Makes GenerationUtils TorchScript-compatible w/ T5 model.

This PR makes the following changes:

Testing:

joecummings commented 1 year ago

@Nayef211 Thoughts on the with_generation_utils on the model bundle versus having this as a separate class to import?

Nayef211 commented 1 year ago

@Nayef211 Thoughts on the with_generation_utils on the model bundle versus having this as a separate class to import?

I think the with_generation_utils is probably a cleaner way to do it and reduces code duplication. I would just make sure to clearly document this parameter so users understand what it does and how it's used.

joecummings commented 1 year ago

@atalman @osalpekar Why is torchdata no longer being picked up in smoke_tests.py?