search

argmaxinc / WhisperKit

On-device Speech Recognition for Apple Silicon
http://argmaxinc.com/blog/whisperkit
MIT License
3.87k stars 329 forks source link

MLX #33

Open fakerybakery opened 8 months ago

fakerybakery commented 8 months ago

Hey, Apple just dropped MLX-Swift, a cross-platform (currently only iOS/macOS) MLX framework. Are there any plans to support it? Thanks!

atiorh commented 8 months ago

Stay tuned for an announcement soon :)

ZachNagengast commented 8 months ago

Worth noting here that MLX doesn't yet support ANE, but there are a number of tasks in WhisperKit that it is a great fit for 🚀

vade commented 8 months ago

Now that MLX supports Swift, in theory, custom CoreML layers could be implemented in MLX - and assuming one can configure the MLX layers to operate on float 16 and have an IOSurface backed memory storage (which, I believe, unless I am under a misapprehension MLX Supports and MLX Swift should as well?) you could have zero copy from buffers computed in ANE <-> MLX Layers <-> ANE

Which would be super cool