[ ] Bug fix (non-breaking change which fixes an issue)
[x] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] New feature (non-breaking change which adds functionality)
[ ] This change requires a documentation update
Feature/Issue validation/testing
Please describe the Unit or Integration tests that you ran to verify your changes and relevant result summary. Provide instructions so it can be reproduced.
Please also list any relevant details for your test configuration.
pytest -v test_torch_compile.py
================================================================================== test session starts ===================================================================================
platform linux -- Python 3.10.14, pytest-7.3.1, pluggy-1.0.0 -- /home/ubuntu/anaconda3/envs/torchserve/bin/python
cachedir: .pytest_cache
rootdir: /home/ubuntu/serve
plugins: cov-4.1.0, mock-3.14.0, timeout-2.3.1
collected 8 items
test_torch_compile.py::TestTorchCompile::test_archive_model_artifacts PASSED [ 12%]
test_torch_compile.py::TestTorchCompile::test_start_torchserve PASSED [ 25%]
test_torch_compile.py::TestTorchCompile::test_server_status PASSED [ 37%]
test_torch_compile.py::TestTorchCompile::test_registered_model PASSED [ 50%]
test_torch_compile.py::TestTorchCompile::test_serve_inference SKIPPED (Test failing on regression runner) [ 62%]
test_torch_compile.py::TestTorchCompile::test_compile_inference_enable_true_default PASSED [ 75%]
test_torch_compile.py::TestTorchCompile::test_compile_inference_enable_true PASSED [ 87%]
test_torch_compile.py::TestTorchCompile::test_compile_inference_enable_false PASSED [100%]
==================================================================================== warnings summary ====================================================================================
test_torch_compile.py:12
/home/ubuntu/serve/test/pytest/test_torch_compile.py:12: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
from pkg_resources import packaging
../../../anaconda3/envs/torchserve/lib/python3.10/site-packages/pkg_resources/__init__.py:2832
../../../anaconda3/envs/torchserve/lib/python3.10/site-packages/pkg_resources/__init__.py:2832
/home/ubuntu/anaconda3/envs/torchserve/lib/python3.10/site-packages/pkg_resources/__init__.py:2832: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('ruamel')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
declare_namespace(pkg)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
======================================================================= 7 passed, 1 skipped, 3 warnings in 35.29s ========================================================================
Checklist:
[ ] Did you have fun?
[ ] Have you added tests that prove your fix is effective or that this feature works?
[ ] Has code been commented, particularly in hard-to-understand areas?
[ ] Have you made corresponding changes to the documentation?
Description
This PR standardizes how TorchServe would be supporting any PyTorch 2.x APIs config
Define the following structure for a PyTorch 2.x feature config in
model-config.yaml
or
For
torch.compile
, we would specify the following optionsThis PR also fixes the issues with torch compile pytests
Fixes #( issue1 issue2 issue3 )
Type of change
Please delete options that are not relevant.
Feature/Issue validation/testing
Please describe the Unit or Integration tests that you ran to verify your changes and relevant result summary. Provide instructions so it can be reproduced. Please also list any relevant details for your test configuration.
Checklist: