Draft PR for the early stages of supporting MLX based whisper models directly in WhisperKit. (To be updated)
Initial TODOs:
[x] Setup swift package structure
[x] MLXFeatureExtractor using FeatureExtracting protocol
[x] MLXAudioEncoder inference using AudioEncoding protocol
[x] MLXTextDecoder inference using TextDecoding protocol or TextDecoder subclass (research needed)
[x] Downloading MLX models (research needed to mix and match model downloads between CoreML and MLX)
[x] Loading MLX models with a new model protocol
[x] Pipeline allowing mixing CoreML and MLX models
[x] ~Compiling whisperkit-cli optionally with/without MLX~ Crossing this out for now: whisperkit-cli does not have the same bundle size constraints as the core library
[x] CI unit tests for iOS platform using xcodebuild instead of iOS simulator (simulator is not supported for MLX)
Draft PR for the early stages of supporting MLX based whisper models directly in WhisperKit. (To be updated)
Initial TODOs:
FeatureExtracting
protocolAudioEncoding
protocolTextDecoding
protocol orTextDecoder
subclass (research needed)xcodebuild
instead of iOS simulator (simulator is not supported for MLX)