argmaxinc / WhisperKit

On-device Speech Recognition for Apple Silicon
http://argmaxinc.com/blog/whisperkit
MIT License
3.92k stars 330 forks source link

WhisperAX demo doesn't copy with streaming #173

Closed iGerman00 closed 4 months ago

iGerman00 commented 4 months ago

Recently it was brought to my attention that the copy button doesn't really work when streaming or using eager mode. It really should and should also probably be bound to Cmd+C. Me and my friends use it quite often as a quick way to live transcribe something. I've modified it locally but thought I'd bring it to your attention, too. It would also be really nice to get an updated TestFlight build every now and again.

Proposition:

.toolbar(content: {
                ToolbarItem {
                    Button {
                        if (!enableEagerDecoding) {
                            let fullTranscript = formatSegments(confirmedSegments + unconfirmedSegments, withTimestamps: enableTimestamps).joined(separator: "\n")
                            #if os(iOS)
                            UIPasteboard.general.string = fullTranscript
                            #elseif os(macOS)
                            NSPasteboard.general.clearContents()
                            NSPasteboard.general.setString(fullTranscript, forType: .string)
                            #endif
                        } else {
                            #if os(iOS)
                            UIPasteboard.general.string = confirmedText + hypothesisText
                            #elseif os(macOS)
                            NSPasteboard.general.clearContents()
                            NSPasteboard.general.setString(confirmedText + hypothesisText, forType: .string)
                            #endif
                        }
                    } label: {
                        Label("Copy Text", systemImage: "doc.on.doc")
                    }
                    .keyboardShortcut("c", modifiers: .command)
                    .foregroundColor(.primary)
                    .frame(minWidth: 0, maxWidth: .infinity)
                }
            })
        }
ZachNagengast commented 4 months ago

Good callout. This makes sense because the eager mode doesn't use these segments internally. It's definitely a TODO for the sample app. As a workaround, we recently added the ability to select the text manually with your cursor now.

ZachNagengast commented 4 months ago

@iGerman00 Do you want to submit this proposal as a PR?

iGerman00 commented 4 months ago

If the code looks acceptable to you, sure. I'm not very familiar with Xcode/Swift

ZachNagengast commented 4 months ago

Resolved by #178