search

ggerganov / whisper.cpp

Port of OpenAI's Whisper model in C/C++
MIT License
35.95k stars 3.67k forks source link

Issue while running wisper.objc #1822

Open raogsm7 opened 10 months ago

raogsm7 commented 10 months ago

This one ran smoothly before upgrading to Sonoma on Mac with an Intel chip on it. Now with Sonoma IOS, it is giving me the following error. Can you please guide me on this issue? Thank you! =====> 2024-02-01 11:16:07.288036-0800 whisper.objc[9734:137428] [AQ] AudioQueueObject.cpp:1580 BuildConverter: AudioConverterNew returned -50 from: 0 ch, 16000 Hz, .... (0x00000000) 0 bits/channel, 0 bytes/packet, 0 frames/packet, 0 bytes/frame to: 1 ch, 16000 Hz, Int16 2024-02-01 11:16:07.288337-0800 whisper.objc[9734:137619] [AQ] AudioQueueObject.cpp:6643 ConvertInput: aq@0x112097000: AudioConverterFillComplexBuffer returned -50, packetCount 8000 2024-02-01 11:16:07.964119-0800 whisper.objc[9734:137428] Stop capturing 2024-02-01 11:16:07.964781-0800 whisper.objc[9734:137428] Not capturing, ignoring audio 2024-02-01 11:16:07.964874-0800 whisper.objc[9734:137428] Not capturing, ignoring audio 2024-02-01 11:16:07.964954-0800 whisper.objc[9734:137428] Not capturing, ignoring audio 2024-02-01 11:55:47.968021-0800 whisper.objc[9734:137620] HALB_MachPort.cpp:174 HALB_MachPort::DestroyPort: not going to destroy anything this way

jahrichie commented 9 months ago

Also hitting this, running in the simulator, 15 max pro, iOS 17, any ideas?

Start capturing
     AudioQueueObject.cpp:1580  BuildConverter: AudioConverterNew returned -50
 from:  0 ch,  16000 Hz, .... (0x00000000) 0 bits/channel, 0 bytes/packet, 0 frames/packet, 0 bytes/frame
to:    1 ch,  16000 Hz, Int16
     AudioQueueObject.cpp:6707  ConvertInput: aq@0x14c00aa00: AudioConverterFillComplexBuffer returned -50, packetCount 8000
[C:2] Error received: Connection interrupted.
[C:2-1] Error received: Connection interrupted.
-[RTIInputSystemClient _configureConnection:withMachName:]_block_invoke  Client connection to service was interrupted: <NSXPCConnection: 0x60000333db80> connection to service with pid -1 named (null)
Stop capturing
Not capturing, ignoring audio
Not capturing, ignoring audio
Not capturing, ignoring audio
Start capturing
     AudioQueueObject.cpp:1580  BuildConverter: AudioConverterNew returned -50
 from:  0 ch,  16000 Hz, .... (0x00000000) 0 bits/channel, 0 bytes/packet, 0 frames/packet, 0 bytes/frame
to:    1 ch,  16000 Hz, Int16
Stop capturing
Not capturing, ignoring audio
Not capturing, ignoring audio
Not capturing, ignoring audio
Start capturing
     AudioQueueObject.cpp:1580  BuildConverter: AudioConverterNew returned -50
 from:  0 ch,  16000 Hz, .... (0x00000000) 0 bits/channel, 0 bytes/packet, 0 frames/packet, 0 bytes/frame
to:    1 ch,  16000 Hz, Int16
     AudioQueueObject.cpp:6707  ConvertInput: aq@0x15580ee00: AudioConverterFillComplexBuffer returned -50, packetCount 8000
Stop capturing
Not capturing, ignoring audio
Not capturing, ignoring audio
Not capturing, ignoring audio
Start capturing
     AudioQueueObject.cpp:1580  BuildConverter: AudioConverterNew returned -50
 from:  0 ch,  16000 Hz, .... (0x00000000) 0 bits/channel, 0 bytes/packet, 0 frames/packet, 0 bytes/frame
to:    1 ch,  16000 Hz, Int16
Stop capturing
Not capturing, ignoring audio
Not capturing, ignoring audio
Not capturing, ignoring audio
Start capturing
     AudioQueueObject.cpp:1580  BuildConverter: AudioConverterNew returned -50
 from:  0 ch,  16000 Hz, .... (0x00000000) 0 bits/channel, 0 bytes/packet, 0 frames/packet, 0 bytes/frame
to:    1 ch,  16000 Hz, Int16
Stop capturing
Not capturing, ignoring audio
Not capturing, ignoring audio
Not capturing, ignoring audio
Start capturing
     AudioQueueObject.cpp:1580  BuildConverter: AudioConverterNew returned -50
 from:  0 ch,  16000 Hz, .... (0x00000000) 0 bits/channel, 0 bytes/packet, 0 frames/packet, 0 bytes/frame
to:    1 ch,  16000 Hz, Int16
Stop capturing
Not capturing, ignoring audio
Not capturing, ignoring audio
Not capturing, ignoring audio
Start capturing
     AudioQueueObject.cpp:1580  BuildConverter: AudioConverterNew returned -50
 from:  0 ch,  16000 Hz, .... (0x00000000) 0 bits/channel, 0 bytes/packet, 0 frames/packet, 0 bytes/frame
to:    1 ch,  16000 Hz, Int16
Stop capturing
Not capturing, ignoring audio
Not capturing, ignoring audio
Not capturing, ignoring audio
Realtime: ON
Realtime: OFF
Realtime: ON
127495          HALC_ProxyIOContext.cpp:1328  HALC_ProxyIOContext::IOWorkLoop: skipping cycle due to overload
Found no UIEvent for backing event of type: 11; contextId: 0xD1876E74
Found no UIEvent for backing event of type: 11; contextId: 0xD1876E74
Found no UIEvent for backing event of type: 11; contextId: 0xD1876E74
Found no UIEvent for backing event of type: 11; contextId: 0xD1876E74
Found no UIEvent for backing event of type: 11; contextId: 0xD1876E74
Found no UIEvent for backing event of type: 11; contextId: 0xD1876E74
Found no UIEvent for backing event of type: 11; contextId: 0xD1876E74
Found no UIEvent for backing event of type: 11; contextId: 0xD1876E74
AvivSham commented 3 months ago

Hi All, We are running into the same issue, it looks like the audio is captured and processed but not printed. here are the CL prints:

Start capturing
Captured 7168 new samples
Captured 8000 new samples
Captured 8000 new samples
Captured 8000 new samples
Captured 8000 new samples
Captured 8000 new samples
Captured 7277 new samples
Captured 7808 new samples
Captured 7168 new samples
Captured 7168 new samples
Captured 7808 new samples
Captured 7168 new samples
Captured 7808 new samples
Captured 7168 new samples
Captured 7808 new samples
Captured 7168 new samples
Captured 7808 new samples
Captured 8000 new samples
Captured 8000 new samples
Captured 8000 new samples
Stop capturing
Not capturing, ignoring audio
Not capturing, ignoring audio
Not capturing, ignoring audio
Processing 153325 samples

whisper_print_timings:     load time =    94.29 ms
whisper_print_timings:     fallbacks = 110 p /   0 h
whisper_print_timings:      mel time =    16.75 ms
whisper_print_timings:   sample time =     4.98 ms /     1 runs (    4.98 ms per run)
whisper_print_timings:   encode time =   122.98 ms /     1 runs (  122.98 ms per run)
whisper_print_timings:   decode time =    55.89 ms /     6 runs (    9.31 ms per run)
whisper_print_timings:   batchd time =     0.00 ms /     1 runs (    0.00 ms per run)
whisper_print_timings:   prompt time =     0.00 ms /     1 runs (    0.00 ms per run)
whisper_print_timings:    total time =   205.60 ms

Processing time: 0.206, on 6 threads
Start capturing
Captured 7168 new samples
Captured 7168 new samples
Captured 7675 new samples
Captured 7168 new samples
Captured 7808 new samples
Captured 7168 new samples
Captured 7808 new samples
Captured 7168 new samples
Captured 7808 new samples
Captured 7168 new samples
Captured 7808 new samples
Captured 7168 new samples
Captured 7808 new samples
Captured 8000 new samples
Captured 8000 new samples
Captured 8000 new samples
Stop capturing
Not capturing, ignoring audio
Not capturing, ignoring audio
Not capturing, ignoring audio
Processing 120891 samples

whisper_print_timings:     load time =    94.29 ms
whisper_print_timings:     fallbacks = 115 p /   0 h
whisper_print_timings:      mel time =    14.41 ms
whisper_print_timings:   sample time =     4.23 ms /     1 runs (    4.23 ms per run)
whisper_print_timings:   encode time =   114.93 ms /     1 runs (  114.93 ms per run)
whisper_print_timings:   decode time =    45.47 ms /     6 runs (    7.58 ms per run)
whisper_print_timings:   batchd time =     0.00 ms /     1 runs (    0.00 ms per run)
whisper_print_timings:   prompt time =     0.00 ms /     1 runs (    0.00 ms per run)
whisper_print_timings:    total time =   182.70 ms

Xcode runs on Mac M2 with OS Sonoma 14.5 The app is installed on iPhone 15 Pro with 17.5.1 iOS version.

@ggerganov can you please help solve this issue? @jahrichie, @raogsm7 - have you found a solution?