Test and improve apple support - Githubissues

nod-ai / serve

Serve, optimize and scale PyTorch models in production

https://pytorch.org/serve/

Apache License 2.0

1 stars 0 forks source link

Test and improve apple support #12

Closed eppane closed 1 week ago

eppane commented 2 weeks ago

Currently TorchServe has support for Apple GPUs:

docs/apple_silicon_support.md

For GPU jobs on Apple Silicon, MPS is now auto detected and enabled. To prevent TorchServe from using MPS, users have to set deviceType: "cpu" in model-config.yaml.
This is an experimental feature and NOT ALL models are guaranteed to work.
Number of GPUs now reports GPUs on Apple Silicon
Pytests check for MPS on MacOS M1 devices
Models that have been tested and work: Resnet-18, Densenet161, Alexnet
Models that have been tested and DO NOT work: MNIST

Action points:

Ensure support for newer chips from the M-series (e.g. M2 and M3)
- See system_metrics.py & print_env_info.py
Investigate limitations of model support and overcome them if possible

eppane commented 2 weeks ago

Some notes: Metal Performance Shaders (MPS) How to check mps availability? torch.backends — PyTorch 2.5 documentation

https://pytorch.org/docs/stable/_modules/torch/backends/mps.html#is_available torch-mps/TorchMPS-Example1.ipynb at main · psuarezserrato/torch-mps

Accelerated PyTorch training on Mac - Metal - Apple Developer Tensor Attributes — PyTorch 2.5 documentation

jakki-amd commented 1 week ago

Won't fix as not urgent.