Open ivarflakstad opened 10 months ago
@cwfitzgerald (Sorry for the ping if you're not the correct person).
We're iterating quite a bit on this here -> https://github.com/ivarflakstad/metal-rs We would love some feedback if upstreaming here makes sense or not.
I'm not really involved in metal-rs development outside of administration and releases, so not really sure what fits and what doesn't :)
Fair enough 😊 Maybe you know who does?
@kvark Do you know who might be involved with metal-rs?
@Wumpf Maybe ?
After having matured a bit this and implemented the Matrix
in https://github.com/huggingface/candle/ I feel like this PR should be trimmed down to only get the MPSMatrix
support and remove the rest of the sugar (being more aligned with the rest of the crate).
Ofc having maintainer advice would be helpful.
Also in order to be able to publish on our side, we're likely going to fork metal-rs
for the time being to have access to those objects.
We might return to metal-rs
once we get rid of MPSMatrix
dependency (if we can using metal-flash-attention, which seems faster, but currently isn't as widely portable as MPSMatrix since it doesn't work on first gen m1).
Either that or when this PR gets included.
Edit: outdated, see https://github.com/ivarflakstad/metal-rs for current state.
Leaving open for visibility.
Hey! Gave this a go.
Output from running
cargo run --release --example mps-matrix-multiplication --features=mps