Closed zfarrell13 closed 9 months ago
Did you have any luck with this? I'm also interested. I'm fairly new to GPU programming, but it seems like it would probably be better/easier to leverage the Metal GPU than to get rid of the GPU support entirely, no? i.e., switch from the Cuda functions to the Metal functions: https://github.com/ggerganov/llama.cpp/blob/master/ggml-metal.m
hey zane, i did end up getting it to work on mac m1, and then later in linux based docker using linux/arm64 machine. it was very involved. I had to build tensorflow-text, jax, and jaxlib from source for my python version (3.8) and machine aarch64. couldnt find a better way. the dockerfile is 2.1gb :(.
I also needed to make quite a few modifications to mt3 and t5x setup.py files and scripts in order to use different versions of libraries only available on python 3.8.
Would you mind sharing the Dockerfile? I’ve been piecing one together myself but was in the middle of dealing with the versioning hell that is all of Python, as you alluded.
Im trying to build mt3 execution into a dockerfile on my M1 mac. i know this is involved and im willing to modify the code to exclude any reference to gpu required lines of code/jax. it is a lost cause however, if the checkpoints are not executable via cpu. is that the case?