Export transform and model for torchscript, quantization and deployment

facebookresearch / recipes

Recipes are a standard, well supported set of blueprints for machine learning engineers to rapidly train models using the latest research techniques without significant engineering overhead.Specifically, recipes aims to provide- Consistent access to pre-trained SOTA models ready for production- Reference implementations for SOTA research reproducibility, and infrastructure to guarantee correctness, efficiency, and interoperability.

BSD 3-Clause "New" or "Revised" License

284 stars 35 forks source link

Export transform and model for torchscript, quantization and deployment #36

Closed hudeven closed 1 year ago

hudeven commented 1 year ago

It's troublesome to deal with transform and model separately during torchscript, quantization and deploy with torchserve. So I added CombinedModule wrapper to hold them such that it can consume raw input(text) and produce the prediction(generated text) end to end.

facebook-github-bot commented 1 year ago

@hudeven has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.