search

pytorch / serve

Serve, optimize and scale PyTorch models in production
https://pytorch.org/serve/
Apache License 2.0
4.04k stars 821 forks source link

Fix Inf2 benchmark #3177

Closed namannandan closed 3 weeks ago

namannandan commented 3 weeks ago

Description

Inf2 benchmarks have been failing due to the following reasons:

  1. The latest version of torch-neuronx requires torch 2.1.2 and torch was pinned to 1.13
  2. Although setuptools is pinned, it was getting upgraded when loading the benchmark models due to installation of custom dependencies ex: torch-neuronx. Related issue: https://github.com/pytorch/serve/issues/3176

Fix:

  1. Pin torch to a know supported version for neuronx
  2. Update Inf2 model artifacts to not use custom dependencies since these are already setup by install_dependencies.py script before running the benchmark.

Type of change

Please delete options that are not relevant.

Feature/Issue validation/testing