Torchserve support mac M1 chip

pytorch / serve

Serve, optimize and scale PyTorch models in production

https://pytorch.org/serve/

Apache License 2.0

4.23k stars 864 forks source link

Torchserve support mac M1 chip #1678

Open lxning opened 2 years ago

lxning commented 2 years ago

🚀 The feature

According to PyTorch installation instruction, Torchserve needs to update PyTorch installation and docker for Mac M1.

"MacOS Conda binaries are for x86_64 only, for M1 please use wheels conda install pytorch torchvision torchaudio -c pytorch"

Motivation, pitch

To unblock TS on Mac M1.

Alternatives

No response

Additional context

This ticket will cover #1363 and #1673

lsmith77 commented 1 year ago

anything happening here? I am unable to get TorchServe to run (keep getting "Backend worker error" on startup) and for Docker I guess M1 compatible images are missing.

msaroufim commented 1 year ago

M1 compatible images can be created using https://github.com/pytorch/serve/issues/1363 - other than that we'd probably need to also make sure base_handler.py also sends tensors to M1 devices if available - happy to advise if you to send a PR

msaroufim commented 1 year ago

Wrote a more detailed proposal here https://github.com/pytorch/serve/issues/2555