Image for arm64 (Macbook Pro) - Githubissues

huggingface / text-generation-inference

Large Language Model Text Generation Inference

http://hf.co/docs/text-generation-inference

Apache License 2.0

9.02k stars 1.06k forks source link

Image for arm64 (Macbook Pro) #2560

Open arsentievalex opened 1 month ago

arsentievalex commented 1 month ago

System Info

Which image should I use on Macbook Pro? I can't find arm64 image. Please see the below error I'm having:

1 warning found (use docker --debug to expand):
 - InvalidBaseImagePlatform: Base image ghcr.io/huggingface/text-generation-inference:sha-f3c5d7d@sha256:7c169c536de65dbdb867302aedf9ad0a7472df5d7c7dfd85c080037215b18c16 was pulled with platform "linux/amd64", expected "linux/arm64" for current build (line 1)
Containerfile:67

I'm trying to install it using NVIDIA AI Workbench.

Information

[X] Docker
[ ] The CLI directly

Tasks

[ ] An officially supported command
[ ] My own modifications

Reproduction

Clone https://github.com/NVIDIA/workbench-example-hybrid-rag using NVIDIA AI Workbench on Macbook Pro

Expected behavior

The environment is built successfully.

trainerbox commented 3 weeks ago

TGI is not hardware compatible with Mac silicon https://github.com/huggingface/text-generation-inference

Hardware support [Nvidia] [AMD] [Inferentia] [Gaudi] [Google TPU]

Additional ref: https://www.datacamp.com/tutorial/hugging-faces-text-generation-inference-toolkit-for-llms