argmaxinc / WhisperKit

On-device Speech Recognition for Apple Silicon
http://argmaxinc.com/blog/whisperkit
MIT License
3.98k stars 338 forks source link

"Resource path does not exist" Error #210

Open mintflavourweb opened 2 months ago

mintflavourweb commented 2 months ago

I am developing an app using WhisperKit and SwiftUI for iPhone.

I keep getting this error while using the transcribe method: Resource path does not exist file:///Users/{myuser}/Library/Developer/CoreSimulator/Devices/BF68DAF2-55B5-43A2-B0B1-03969ACAB6A8/data/Containers/Data/Application/21A29C2D-BFB3-41E9-9821-3D464901F1A5/Documents/audio.mp3

I'm sure the file exists since I can see it in the folder. The file has been copied and moved to the Documents folder after being picked with fileImporter, like so:

func moveFileToAppDirectory(sourceURL: URL) -> URL? {
        let fileManager = FileManager.default

        // get permissions
        if sourceURL.startAccessingSecurityScopedResource() {
            defer { sourceURL.stopAccessingSecurityScopedResource() }

            // force iCloud download
            if fileManager.isUbiquitousItem(at: sourceURL) {
                do {
                    try fileManager.startDownloadingUbiquitousItem(at: sourceURL)
                    print("File downloaded from iCloud.")
                } catch {
                    print("Error downloading from iCloud: \(error.localizedDescription)")
                    return nil
                }
            }

            let fileExtension = sourceURL.pathExtension

            // get document path
            if let documentsDirectory = fileManager.urls(for: .documentDirectory, in: .userDomainMask).first {
                // create new path
                let destinationURL = documentsDirectory.appendingPathComponent("audio.\(fileExtension)")

                do {
                    // If it already exists, remove it
                    if fileManager.fileExists(atPath: destinationURL.path) {
                        try fileManager.removeItem(at: destinationURL)
                        print("File removed: \(destinationURL)")
                    }
                    // copy in the document folder
                    try fileManager.copyItem(at: sourceURL, to: destinationURL)
                    print("File copied in \(destinationURL)")
                    return destinationURL
                } catch {
                    print("Error: \(error.localizedDescription)")
                    return nil
                }
            }
        } else {
            print("No permission given to access the file")
        }
        return nil
    }

I handle the file pick in this way:

func handleFileMediaResponse(_ result: Result<URL, any Error>) async {
        selectedAudio = nil
            switch result {
                case .success(let success):
                    // move file to private documents folder
                    if let newFileURL = moveFileToAppDirectory(sourceURL: success) {
                        selectedAudio = newFileURL
                        // download whisper model
                        let folder = try? await WhisperKit.download(variant: "base", progressCallback: { progress in
                            downloadProgressValue = Float(progress.fractionCompleted)
                            print(downloadProgressValue)
                        })
                        let pipe = try? await WhisperKit(modelFolder: folder?.absoluteString, verbose: true, logLevel: .debug)
                        print("pipe created")

                        // transcribe
                        let transcribeResult = await pipe!.transcribeWithResults(audioPaths: [selectedAudio?.absoluteString ?? ""])
                        var result = [Result<[TranscriptionResult], Swift.Error>]()
                        for res in transcribeResult {
                            result.append(res)

                            switch res {
                            case .success(let transcriptionResults):
                                print("Transcription Results: \(transcriptionResults)")
                            case .failure(let error):
                                print("Transcription Error: \(error.localizedDescription)")
                            }
                        }
                    } else {
                        print("Error while moving file.")
                    }
                case .failure(let failure):
                    print("Error: \(failure.localizedDescription)")
            }
    }

In the console I get this:

    File copied in 
    file:///Users/{myuser}/Library/Developer/CoreSimulator/Devices/BF68DAF2-55B5-43A2-B0B1-03969ACAB6A8/data/Containers/Data/Application/21A29C2D-BFB3-41E9-9821-3D464901F1A5/Documents/audio.mp3

Entire log:

1.0

Running on arm64

Loading models...

Loading models from /file:/Users/{my_user}/Library/Developer/CoreSimulator/Devices/BF68DAF2-55B5-43A2-B0B1-03969ACAB6A8/data/Containers/Data/Application/21A29C2D-BFB3-41E9-9821-3D464901F1A5/Documents/huggingface/models/argmaxinc/whisperkit-coreml/openai_whisper-base with prewarmMode: false

Loading feature extractor

Failed to get the home directory when checking model path.

Loaded feature extractor

Loading text decoder

Failed to get the home directory when checking model path.

Loaded text decoder in 0.37s

Loading audio encoder

Failed to get the home directory when checking model path.

Loaded audio encoder in 0.36s

Loading tokenizer for base

Loading tokenizer from local folder

Loaded tokenizer in 0.20s

Loaded models for whisper size: base in 1.06s

pipe created

Total Audio Loading and Converting Time: 0.0007600784301757812

Resource path does not exist file:///Users/{my_user}/Library/Developer/CoreSimulator/Devices/BF68DAF2-55B5-43A2-B0B1-03969ACAB6A8/data/Containers/Data/Application/21A29C2D-BFB3-41E9-9821-3D464901F1A5/Documents/audio.mp3

Transcription Error: Resource path does not exist file:///Users/{my_user}/Library/Developer/CoreSimulator/Devices/BF68DAF2-55B5-43A2-B0B1-03969ACAB6A8/data/Containers/Data/Application/21A29C2D-BFB3-41E9-9821-3D464901F1A5/Documents/audio.mp3

Any help is appreciated! Thank you.

latenitefilms commented 2 months ago

My GUESS is that you're triggering stopAccessingSecurityScopedResource, so WhisperKit doesn't have sandbox access to the file?