pytorch / serve

Serve, optimize and scale PyTorch models in production
https://pytorch.org/serve/
Apache License 2.0
4.04k stars 821 forks source link

torch compile config standardization update #3166

Closed agunapal closed 2 weeks ago

agunapal commented 4 weeks ago

Description

This PR standardizes how TorchServe would be supporting any PyTorch 2.x APIs config

Define the following structure for a PyTorch 2.x feature config in model-config.yaml

pt2:
  <API name>:
    enable: True
    option1:  value1
    optionN:  valueN

or

pt2:
  <API name>:
    <feature 1>:
      enable: True
      option1:  value1
      optionN:  valueN

For torch.compile, we would specify the following options

pt2:
  compile:
    enable: True
    backend: "inductor"
    mode: "max-autotune"

This PR also fixes the issues with torch compile pytests

Fixes #( issue1 issue2 issue3 )

Type of change

Please delete options that are not relevant.

Feature/Issue validation/testing

Please describe the Unit or Integration tests that you ran to verify your changes and relevant result summary. Provide instructions so it can be reproduced. Please also list any relevant details for your test configuration.

pytest -v test_torch_compile.py
================================================================================== test session starts ===================================================================================
platform linux -- Python 3.10.14, pytest-7.3.1, pluggy-1.0.0 -- /home/ubuntu/anaconda3/envs/torchserve/bin/python
cachedir: .pytest_cache
rootdir: /home/ubuntu/serve
plugins: cov-4.1.0, mock-3.14.0, timeout-2.3.1
collected 8 items                                                                                                                                                                        

test_torch_compile.py::TestTorchCompile::test_archive_model_artifacts PASSED                                                                                                       [ 12%]
test_torch_compile.py::TestTorchCompile::test_start_torchserve PASSED                                                                                                              [ 25%]
test_torch_compile.py::TestTorchCompile::test_server_status PASSED                                                                                                                 [ 37%]
test_torch_compile.py::TestTorchCompile::test_registered_model PASSED                                                                                                              [ 50%]
test_torch_compile.py::TestTorchCompile::test_serve_inference SKIPPED (Test failing on regression runner)                                                                          [ 62%]
test_torch_compile.py::TestTorchCompile::test_compile_inference_enable_true_default PASSED                                                                                         [ 75%]
test_torch_compile.py::TestTorchCompile::test_compile_inference_enable_true PASSED                                                                                                 [ 87%]
test_torch_compile.py::TestTorchCompile::test_compile_inference_enable_false PASSED                                                                                                [100%]

==================================================================================== warnings summary ====================================================================================
test_torch_compile.py:12
  /home/ubuntu/serve/test/pytest/test_torch_compile.py:12: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    from pkg_resources import packaging

../../../anaconda3/envs/torchserve/lib/python3.10/site-packages/pkg_resources/__init__.py:2832
../../../anaconda3/envs/torchserve/lib/python3.10/site-packages/pkg_resources/__init__.py:2832
  /home/ubuntu/anaconda3/envs/torchserve/lib/python3.10/site-packages/pkg_resources/__init__.py:2832: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('ruamel')`.
  Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
    declare_namespace(pkg)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
======================================================================= 7 passed, 1 skipped, 3 warnings in 35.29s ========================================================================

Checklist: