search

milvus-io / milvus

A cloud-native vector database, storage for next generation AI applications
https://milvus.io
Apache License 2.0
28.53k stars 2.75k forks source link

[Feature]: Apple Silicone GPU (Metal Framework) & Apple Nural Engine (ANE) #32791

Open qdrddr opened 2 months ago

qdrddr commented 2 months ago

Is there an existing issue for this?

Is your feature request related to a problem? Please describe.

Milvus can utilize NVIDIA/CUDA to enhance processing speed with GPU. Having Milvus running on M1/M2/M3/M4 ARM arhitecture can speedup development as many developers are using Apple macOS platform.

Describe the solution you'd like.

Run Milvus locally on macOS ARM64 architecture and utilize GPU (via Metal Framework) and ANE accelerators.

Describe an alternate solution.

Runing on servers with nvidia or CPU-only processing.

Anything else? (Additional Context)

No response

alexanderguzhva commented 2 months ago

https://developer.apple.com/documentation/apple-silicon/cpu-optimization-guide

xiaofan-luan commented 2 months ago

@qdrddr what is the use case? running vector db on laptop doesn't really make sense.

xiaofan-luan commented 2 months ago

maybe it's interesting to run milvus-lite on apple GPUs

alexanderguzhva commented 2 months ago

@xiaofan-luan it might make sense in the future, though

https://appleinsider.com/articles/24/05/07/secret-apple-project-acdc-to-pioneer-ai-chips-for-data-centers

Under the internal name "Project ACDC," Apple is developing Apple Silicon designed specifically for server farms dedicated to AI processing. The company aims to optimize AI applications within its data centers for future versions of its platforms.

qdrddr commented 2 months ago

@qdrddr what is the use case? running vector db on laptop doesn't really make sense.

What If it's a Maximum config of MaxStudio or Mac Pro with M2 Ultra and 196GB RAM available for GPU with 60 cores?