`detectLanguage` isn't working

[stasbel]

version: 0.7.2 snippet to reproduce:

import Foundation
import WhisperKit

Task {
  guard let desktopURL = FileManager.default.urls(
    for: .desktopDirectory,
    in: .userDomainMask
  ).first else { return }
  print("desktopURL: \(desktopURL)")

  let model = try await WhisperKit(model: "base")
  try await model.loadModels(prewarmMode: true)

  let fileURL = desktopURL.appendingPathComponent("0.wav")
  print("absoluteString: \(fileURL.path)")
  let (lang, langProbs) = try await model.detectLanguage(audioPath: fileURL.path)
  print("lang: \(lang), langProbs: \(langProbs)")
}

file: 0.wav.zip

tiny version gives:

lang: en, langProbs: ["en": -0.35910118]"

which is not all languages and has negative prob

base version gives:

[WhisperKit] Detected language nocaptions is not supported, defaulting to en
lang: en, langProbs: [:]

which is just plain error somewhere inside whisperkit

[atiorh]

Thanks for the report!

• The negative value is because it is a logProb value which has range [-inf,0]. To ease its usage, we should undo the log so you get [0,1]
• The second issue is an edge case we should fix. When no speech is detected, the token that is supposed to signify the predicted language is not predicting a non0language token. Our TokenSupressFilter can fix this.

[stasbel]

thanks what about predicting multiple languages and corresponding probs? why "no speech is detected"? audio example clearly has some :)

[stasbel]

@ZachNagengast hi, is it on track to be completed anytime soon?

[ZachNagengast]

@stasbel This looks like it was actually a tiny oversight in the filter logic and not related to the sampling issues we just put out a fix for. Please check main with commit https://github.com/argmaxinc/WhisperKit/commit/c93d613c8c6fc3ec5b9d1d9da5d1f4206183c5e4 for the fix.

[ZachNagengast]

@stasbel Checking on this, is your issue resolved with the latest change?