Closed zxl777 closed 8 months ago
atiorh
commented
8 months ago Hi @zxl777, I am setting up a 12 mini with the latest WhisperAX release, i.e. 0.1.2 (2) to inspect this potential memory use issue. Could you also let me know your iOS version?
atiorh
commented
8 months ago I anticipate 1050MB (aggregate model weights) + ~200MB (peak model activation memory) which is well below this device's generally available RAM (2GB out of 4GB physical)
atiorh
commented
8 months ago I was able to reproduce the crash on iOS17, thanks for the report!
In the short term, I recommend sticking to the small and small.en variants on iPhone 12 Mini and we will update the device support to exclude large for the time being (cc: @ZachNagengast). large-v2_1050MB successfully runs on iPhone 12 Pro Max so this rules out a chip generation problem. I suspect 1050MB is just over the edge of critical peak memory we should stay under. We will be releasing 10-20% smaller large-v2 and large-v3 models with better quality within the next 2 weeks. This is highly likely to fix this issue. I will circle back with results on those here.
zxl777
commented
8 months ago Thank you, it works normally when tested on other models.
Thank you for your great work and the value it brings!
atiorh
commented
7 months ago @zxl777 Hey! Do you mind trying out the new variants on https://huggingface.co/argmaxinc/whisperkit-coreml/tree/main? Compressed variants of large-v* are all <1GB now.
zxl777
commented
7 months ago @atiorh Today, I tested the latest version of the WhisperAX app consistently crashes when loading the 'openai_whisper-large-v3_947MB' file. I'm using an iPhone 12 mini and downloaded the ZIP file from GitHub.
The app “WhisperAX” has been killed by the operating system because it is using too much memory.
Domain: IDEDebugSessionErrorDomain
Code: 11
Recovery Suggestion: Use a memory profiling tool to track the process memory usage.
User Info: {
DVTErrorCreationDateKey = "2024-03-26 17:52:03 +0000";
IDERunOperationFailingWorker = DBGLLDBLauncher;
}
--
The app “WhisperAX” has been killed by the operating system because it is using too much memory.
Domain: IDEDebugSessionErrorDomain
Code: 11
Recovery Suggestion: Use a memory profiling tool to track the process memory usage.
User Info: {
IDERunOperationFailingWorker = DBGLLDBLauncher;
}
--
Event Metadata: com.apple.dt.IDERunOperationWorkerFinished : {
"device_isCoreDevice" = 1;
"device_model" = "iPhone13,1";
"device_osBuild" = "17.3.1 (21D61)";
"device_platform" = "com.apple.platform.iphoneos";
"dvt_coredevice_version" = "355.24";
"dvt_mobiledevice_version" = "1643.100.58";
"launchSession_schemeCommand" = Run;
"launchSession_state" = 2;
"launchSession_targetArch" = arm64;
"operation_duration_ms" = 1236498;
"operation_errorCode" = 11;
"operation_errorDomain" = IDEDebugSessionErrorDomain;
"operation_errorWorker" = DBGLLDBLauncher;
"operation_name" = IDERunOperationWorkerGroup;
"param_debugger_attachToExtensions" = 0;
"param_debugger_attachToXPC" = 1;
"param_debugger_type" = 3;
"param_destination_isProxy" = 0;
"param_destination_platform" = "com.apple.platform.iphoneos";
"param_diag_MainThreadChecker_stopOnIssue" = 0;
"param_diag_MallocStackLogging_enableDuringAttach" = 0;
"param_diag_MallocStackLogging_enableForXPC" = 1;
"param_diag_allowLocationSimulation" = 1;
"param_diag_checker_tpc_enable" = 1;
"param_diag_gpu_frameCapture_enable" = 0;
"param_diag_gpu_shaderValidation_enable" = 0;
"param_diag_gpu_validation_enable" = 0;
"param_diag_memoryGraphOnResourceException" = 0;
"param_diag_queueDebugging_enable" = 1;
"param_diag_runtimeProfile_generate" = 0;
"param_diag_sanitizer_asan_enable" = 0;
"param_diag_sanitizer_tsan_enable" = 0;
"param_diag_sanitizer_tsan_stopOnIssue" = 0;
"param_diag_sanitizer_ubsan_stopOnIssue" = 0;
"param_diag_showNonLocalizedStrings" = 0;
"param_diag_viewDebugging_enabled" = 1;
"param_diag_viewDebugging_insertDylibOnLaunch" = 1;
"param_install_style" = 2;
"param_launcher_UID" = 2;
"param_launcher_allowDeviceSensorReplayData" = 0;
"param_launcher_kind" = 0;
"param_launcher_style" = 99;
"param_launcher_substyle" = 8192;
"param_runnable_appExtensionHostRunMode" = 0;
"param_runnable_productType" = "com.apple.product-type.application";
"param_structuredConsoleMode" = 1;
"param_testing_launchedForTesting" = 0;
"param_testing_suppressSimulatorApp" = 0;
"param_testing_usingCLI" = 0;
"sdk_canonicalName" = "iphoneos17.4";
"sdk_osVersion" = "17.4";
"sdk_variant" = iphoneos;
}
--
System Information
macOS Version 14.2.1 (Build 23C71)
Xcode 15.3 (22618) (Build 15E204a)
Timestamp: 2024-03-26T10:52:03-07:00Thanks for testing @zxl777! I also reproduced this earlier. We will have to disable large on 12 Mini for now since my prediction that "<1GB model weightsshould be sufficient" did not hold true. The issue not the runtime but the peak memory consumption during compilation. If you really want to make it work in the short term, you can build WhisperAX from source and point tocpuAndGPUinstead of the defaultcpuAndNeuralEngine` but it will be slower and untested.
atiorh
commented
7 months ago Our plan to address this:
1-) MLX backend that avoids this compilation: https://github.com/argmaxinc/WhisperKit/issues/33 (should be similar in perf to cpuAndGPU with current implementation on iOS)
2-) The model versions that are not supported on a given device are manually selected at the moment. We are going to switch to automatic testing to determine this automatically and iPhone 12 Mini + large-v3_947MB won't be enabled until all tests pass
The app crashes after recording a few seconds of sound. It's being used on an iPhone 12 mini device that has been cold restarted, with Large-v2_1050MB.