search

nomic-ai / gpt4all

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
https://nomic.ai/gpt4all
MIT License
68.97k stars 7.57k forks source link

Support Metal or Vulkan on Intel Macs #1354

Open tonyhughesdesign opened 1 year ago

tonyhughesdesign commented 1 year ago

Feature request

Please add support for Tensorflow-Metal for AMD GPUs it will add powerful previous generation cards, and competitive cards to the 4090.

My machine being a prime example has a 32GB AMD Pro Vega II, and I know there are other cards out there like the Pro Vega II Duo 64GB.

https://developer.apple.com/metal/tensorflow-plugin/

Motivation

Lets democratise GPUS.

Your contribution

If you can add Intel AMD GPU support I'm more than happy, to give my time to design, I'm quite capable.

https://tonyhughesdesign.com

manyoso commented 1 year ago

Does this have a vulkan driver available on Apple/Mac? That's the most important question...

tonyhughesdesign commented 1 year ago

Does this have a vulkan driver available on Apple/Mac? That's the most important question...

From all my research Apple refuse to go down the Vulkan libraries direction in favour of their own Metal implementation.

Basten7 commented 8 months ago

Since 2018 Vulkan is implemented on MacOS.

MoltenVK is a layered implementation of Vulkan 1.2 graphics and compute functionality, that is built on Apple's Metal graphics and compute framework on macOS, iOS, tvOS, and visionOS. MoltenVK allows you to use Vulkan graphics and compute functionality to develop modern, cross-platform, high-performance graphical games and applications, and to run them across many platforms, including macOS, iOS, tvOS, visionOS, Simulators, and Mac Catalyst on macOS 11.0+, and all Apple architectures, including Apple Silicon.

https://vulkan.lunarg.com/doc/sdk/1.3.268.1/mac/getting_started.html https://github.com/KhronosGroup/MoltenVK

dokterbob commented 2 weeks ago

Dup of https://github.com/nomic-ai/gpt4all/issues/2114

cebtenzzre commented 2 weeks ago

Dup of #2114

I'll actually close #2114 and #1337, since Metal will be used after translation either way, and it's just a matter of finding a backend that works. Unfortunately, I don't have an Intel Mac to test on.

FellowTraveler commented 2 weeks ago

Dup of #2114

Unfortunately, I don't have an Intel Mac to test on.

I do.