search

googlesamples / mlkit

A collection of sample apps to demonstrate how to use Google's ML Kit APIs on Android and iOS
Apache License 2.0
3.55k stars 2.93k forks source link

[Bug report] MLKSmartReply suggestRepliesForMessages:completion: 'pointer being freed was not allocated' crash #653

Closed antohisorin closed 1 year ago

antohisorin commented 1 year ago

Describe the bug App is crashing in simulator when setting up GoogleMLKit/SmartReply.

To Reproduce Example Steps to reproduce the behavior in sample app:

  1. call - (void)suggestRepliesForMessages:(NSArray<MLKTextMessage > )messages completion:(MLKSmartReplyCallback)completion

App will crash with:

VOMobile(37481,0x31388c000) malloc: error for object 0x31388aa38: pointer being freed was not allocated VOMobile(37481,0x31388c000) malloc: set a breakpoint in malloc_error_break to debug

Expected behavior Smart reply should work without crashing the simulator

SDK Info:

Smartphone:

Development Environment:

Screenshot 2023-04-04 at 15 52 45
miworking3 commented 1 year ago

Thanks for reporting this. I'm afraid it is a known issue: https://developers.google.com/ml-kit/known-issues we are trying to address it in the next release.

iOS issues Simulators on M1-based Mac are not supported. X86_64 simulator is not supported for Translation 'GoogleMLKit/Translate', '4.0.0' or later and Smart Reply 'GoogleMLKit/SmartReply', '4.0.0' or later.

Idomo commented 1 year ago

Thanks for reporting this. I'm afraid it is a known issue: https://developers.google.com/ml-kit/known-issues we are trying to address it in the next release.

iOS issues Simulators on M1-based Mac are not supported. X86_64 simulator is not supported for Translation 'GoogleMLKit/Translate', '4.0.0' or later and Smart Reply 'GoogleMLKit/SmartReply', '4.0.0' or later.

Is there any ETA about when it'll be supported? What about a workaround?

M1-based Macs are existed for almost 3 years (pretty much like the age of this repo), it should be beyond of enough time to make the proper adjustments to support them...

Idomo commented 1 year ago

Just for a reference, this is my crash report:

Macbook Pro 16", 2021, Apple M1 Pro macOS ‏13.4‏(22F66‏) Xcode 14.3.1 (14E300c) iPad Pro (12.9 inch) (6th generation) simulator, iOS 16.4 Pods:

pod 'GoogleMLKit/Translate', '~> 4.0.0'
pod 'GoogleMLKit/TextRecognition', '~> 4.0.0'

Code:

import MLKit

let translatorOptions = TranslatorOptions(sourceLanguage: defaultSourceLang, targetLanguage: defaultTargetLang)
let translator = Translator.translator(options: translatorOptions)

let conditions = ModelDownloadConditions(
  allowsCellularAccess: false,
  allowsBackgroundDownloading: true
)
translator.downloadModelIfNeeded(with: conditions) { error in
  guard error == nil else { return }

  translator.translate(line.text) { translatedText, error in // <<-- CRASHING HERE
    ...
  }
}

Trace stack:

Warning: Error creating LLDB target at path '.../Xcode/DerivedData/MyApp-apjfsvarapbiizaungrxqvdmekod/Build/Products/Debug-iphonesimulator/MyApp.app'- using an empty LLDB target which can cause slow memory reads from remote devices.
MyApp(34414) MallocStackLogging: could not tag MSL-related memory as no_footprint, so those pages will be included in process footprint - (null)
MyApp(34414) MallocStackLogging: recording malloc and VM allocation stacks using lite mode
2023-07-13 19:39:51.044 MyApp[34414/0x20576d280] [lvl=3] +[MLKITx_CCTClearcutUploader crashIfNecessary] Multiple instances of CCTClearcutUploader were instantiated. Multiple uploaders function correctly but have an adverse affect on battery performance due to lock contention.
2023-07-13 19:39:51.112279+0300 MyApp[34414:15553989] Initialized TensorFlow Lite runtime.
INFO: Initialized TensorFlow Lite runtime.
2023-07-13 19:39:51.119439+0300 MyApp[34414:15553989] Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
2023-07-13 19:39:52.286752+0300 MyApp[34414:15553965] [connection] nw_connection_copy_connected_local_endpoint_block_invoke [C3] Client called nw_connection_copy_connected_local_endpoint on unconnected nw_connection
2023-07-13 19:39:52.286925+0300 MyApp[34414:15553965] [connection] nw_connection_copy_connected_remote_endpoint_block_invoke [C3] Client called nw_connection_copy_connected_remote_endpoint on unconnected nw_connection
2023-07-13 19:39:52.287118+0300 MyApp[34414:15553965] [connection] nw_connection_copy_protocol_metadata_internal_block_invoke [C3] Client called nw_connection_copy_protocol_metadata_internal on unconnected nw_connection
I0000 00:00:1689266393.641627       9 package.cc:69] Load: No checksum: .../CoreSimulator/Devices/C6F324F6-E0EF-4177-BF3E-052CE8CD5EB9/data/Containers/Data/Application/DF62E99E-78F6-4964-9CB7-4C02E5816C96/Library/Application Support/com.google.mlkit.nl.translate/en_nl___MLKIT_DEFAULT/dict.en_nl_25/merged_dict_en_nl_25_both.bin
I0000 00:00:1689266393.671894       9 package.cc:69] Load: No checksum: .../CoreSimulator/Devices/C6F324F6-E0EF-4177-BF3E-052CE8CD5EB9/data/Containers/Data/Application/DF62E99E-78F6-4964-9CB7-4C02E5816C96/Library/Application Support/com.google.mlkit.nl.translate/en_nl___MLKIT_DEFAULT/dict.en_nl_25/merged_dict_en_nl_25_from_nl.bin
I0000 00:00:1689266393.672894       9 package.cc:69] Load: No checksum: .../CoreSimulator/Devices/C6F324F6-E0EF-4177-BF3E-052CE8CD5EB9/data/Containers/Data/Application/DF62E99E-78F6-4964-9CB7-4C02E5816C96/Library/Application Support/com.google.mlkit.nl.translate/en_nl___MLKIT_DEFAULT/dict.en_nl_25/merged_dict_en_nl_25_update.bin
I0000 00:00:1689266393.681003       9 varwidtharray.h:242] mlock'ed varwidtharray data successfully.
I0000 00:00:1689266393.681891       9 varwidtharray.h:242] mlock'ed varwidtharray data successfully.
I0000 00:00:1689266393.682233       9 varwidtharray.h:242] mlock'ed varwidtharray data successfully.
I0000 00:00:1689266393.682422       9 offline_lm_approx_map.cc:47] 3500512 bytes actually loaded.
I0000 00:00:1689266393.683954       9 extended_preprocessor.cc:118] Do in-memory preprocessing
W0000 00:00:1689266393.713146       9 universal_preproc_segment_mobile.cc:37] unknown argument: 'chinese_segmenter'
W0000 00:00:1689266393.713172       9 universal_preproc_segment_mobile.cc:37] unknown argument: 'japanese_segmenter'
W0000 00:00:1689266393.713176       9 universal_preproc_segment_mobile.cc:37] unknown argument: 'korean_segmenter'
W0000 00:00:1689266393.713178       9 universal_preproc_segment_mobile.cc:37] unknown argument: 'thai_segmenter'
I0000 00:00:1689266393.742131       9 offline_feature_functions.cc:107] offline lm loaded for language: ENGLISH
I0000 00:00:1689266393.742883       9 offline_feature_functions.cc:107] offline lm loaded for language: ENGLISH
I0000 00:00:1689266393.743024       9 offline_feature_functions.cc:107] offline lm loaded for language: ENGLISH
I0000 00:00:1689266393.743153       9 offline_feature_functions.cc:107] offline lm loaded for language: ENGLISH
I0000 00:00:1689266393.764349       9 inference_session_encoder_util.cc:81] Creating sessions for 4 pipelines...
I0000 00:00:1689266393.764433       9 inference_session_encoder_util.cc:85] Creating session for pipeline 'encoder_embed_ids_0'...
2023-07-13 19:39:53.769405+0300 MyApp[34414:15554016] Initialized TensorFlow Lite runtime.
INFO: Initialized TensorFlow Lite runtime.
2023-07-13 19:39:53.774075+0300 MyApp[34414:15554016] Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
I0000 00:00:1689266393.779056       9 inference_session_encoder_util.cc:85] Creating session for pipeline 'encoder_swizzled_birnn_0'...
I0000 00:00:1689266393.782457       9 inference_session_encoder_util.cc:85] Creating session for pipeline 'encoder_swizzled_birnn_1'...
I0000 00:00:1689266393.783670       9 inference_session_encoder_util.cc:85] Creating session for pipeline 'encoder_swizzled_birnn_2'...
I0000 00:00:1689266393.784941       9 decoder.cc:112] Creating decoder
I0000 00:00:1689266393.784956       9 inference_session_encoder_util.cc:81] Creating sessions for 3 pipelines...
I0000 00:00:1689266393.784972       9 inference_session_encoder_util.cc:85] Creating session for pipeline 'decoder_init_0'...
I0000 00:00:1689266393.790264       9 inference_session_encoder_util.cc:85] Creating session for pipeline 'decoder_step_embed_ids_0'...
I0000 00:00:1689266393.790724       9 inference_session_encoder_util.cc:85] Creating session for pipeline 'decoder_step_0'...
I0000 00:00:1689266393.793408       9 inference_session_encoder_util.cc:81] Creating sessions for 4 pipelines...
I0000 00:00:1689266393.793434       9 inference_session_encoder_util.cc:85] Creating session for pipeline 'encoder_embed_ids_0'...
I0000 00:00:1689266393.793819       9 inference_session_encoder_util.cc:85] Creating session for pipeline 'encoder_swizzled_birnn_0'...
I0000 00:00:1689266393.794770       9 inference_session_encoder_util.cc:85] Creating session for pipeline 'encoder_swizzled_birnn_1'...
I0000 00:00:1689266393.795834       9 inference_session_encoder_util.cc:85] Creating session for pipeline 'encoder_swizzled_birnn_2'...
I0000 00:00:1689266393.796347       9 decoder.cc:112] Creating decoder
I0000 00:00:1689266393.796358       9 inference_session_encoder_util.cc:81] Creating sessions for 3 pipelines...
I0000 00:00:1689266393.796372       9 inference_session_encoder_util.cc:85] Creating session for pipeline 'decoder_init_0'...
I0000 00:00:1689266393.797627       9 inference_session_encoder_util.cc:85] Creating session for pipeline 'decoder_step_embed_ids_0'...
I0000 00:00:1689266393.797895       9 inference_session_encoder_util.cc:85] Creating session for pipeline 'decoder_step_0'...
I0000 00:00:1689266393.812768       9 extended_preprocessor.cc:118] Do in-memory preprocessing
I0000 00:00:1689266393.895873       9 extended_preprocessor.cc:118] Do in-memory preprocessing
I0000 00:00:1689266393.941620       9 wordpiece_preprocessor.cc:46] Model loaded from pb string. Verifying symbols.
I0000 00:00:1689266393.942958       9 wordpiece_preprocessor.cc:81] Symbols=5998 chars=401
I0000 00:00:1689266393.975480       9 wordpiece_preprocessor.cc:46] Model loaded from pb string. Verifying symbols.
I0000 00:00:1689266393.976349       9 wordpiece_preprocessor.cc:81] Symbols=5998 chars=401
I0000 00:00:1689266394.001490       9 wordpiece_preprocessor.cc:46] Model loaded from pb string. Verifying symbols.
I0000 00:00:1689266394.002028       9 wordpiece_preprocessor.cc:81] Symbols=3998 chars=401
I0000 00:00:1689266394.006721       9 dictionary.cc:266] Dictionary build date: 20191024
I0000 00:00:1689266394.010924       9 package.cc:69] Load: No checksum: .../CoreSimulator/Devices/C6F324F6-E0EF-4177-BF3E-052CE8CD5EB9/data/Containers/Data/Application/DF62E99E-78F6-4964-9CB7-4C02E5816C96/Library/Application Support/com.google.mlkit.nl.translate/en_iw___MLKIT_DEFAULT/dict.en_iw_25/merged_dict_en_iw_25_both.bin
I0000 00:00:1689266394.044101       9 package.cc:69] Load: No checksum: .../CoreSimulator/Devices/C6F324F6-E0EF-4177-BF3E-052CE8CD5EB9/data/Containers/Data/Application/DF62E99E-78F6-4964-9CB7-4C02E5816C96/Library/Application Support/com.google.mlkit.nl.translate/en_iw___MLKIT_DEFAULT/dict.en_iw_25/merged_dict_en_iw_25_from_en.bin
I0000 00:00:1689266394.044928       9 package.cc:69] Load: No checksum: .../CoreSimulator/Devices/C6F324F6-E0EF-4177-BF3E-052CE8CD5EB9/data/Containers/Data/Application/DF62E99E-78F6-4964-9CB7-4C02E5816C96/Library/Application Support/com.google.mlkit.nl.translate/en_iw___MLKIT_DEFAULT/dict.en_iw_25/merged_dict_en_iw_25_update.bin
I0000 00:00:1689266394.048642       9 varwidtharray.h:242] mlock'ed varwidtharray data successfully.
I0000 00:00:1689266394.049216       9 varwidtharray.h:242] mlock'ed varwidtharray data successfully.
I0000 00:00:1689266394.049328       9 varwidtharray.h:242] mlock'ed varwidtharray data successfully.
I0000 00:00:1689266394.049338       9 offline_lm_approx_map.cc:47] 3617528 bytes actually loaded.
I0000 00:00:1689266394.049578       9 extended_preprocessor.cc:118] Do in-memory preprocessing
I0000 00:00:1689266394.053114       9 word_reordering_factory.cc:44] Create component with spec: MobileSortReordering/reorder_mobile_sort_eniw:model_data_address=7ff1793db098:model_data_size=1530647
I0000 00:00:1689266394.055663       9 model-api.cc:46] Registered model under key sort-reordering
I0000 00:00:1689266394.062418       9 sort-reordering.cc:96] Successfully constructed valid SortReordering object
I0000 00:00:1689266394.062804       9 offline_feature_functions.cc:107] offline lm loaded for language: HEBREW
I0000 00:00:1689266394.062945       9 offline_feature_functions.cc:107] offline lm loaded for language: HEBREW
I0000 00:00:1689266394.063067       9 offline_feature_functions.cc:107] offline lm loaded for language: HEBREW
I0000 00:00:1689266394.063185       9 offline_feature_functions.cc:107] offline lm loaded for language: HEBREW
I0000 00:00:1689266394.077502       9 inference_session_encoder_util.cc:81] Creating sessions for 4 pipelines...
I0000 00:00:1689266394.077546       9 inference_session_encoder_util.cc:85] Creating session for pipeline 'encoder_embed_ids_0'...
I0000 00:00:1689266394.078103       9 inference_session_encoder_util.cc:85] Creating session for pipeline 'encoder_swizzled_birnn_0'...
I0000 00:00:1689266394.079365       9 inference_session_encoder_util.cc:85] Creating session for pipeline 'encoder_swizzled_birnn_1'...
I0000 00:00:1689266394.080578       9 inference_session_encoder_util.cc:85] Creating session for pipeline 'encoder_swizzled_birnn_2'...
I0000 00:00:1689266394.081186       9 decoder.cc:112] Creating decoder
I0000 00:00:1689266394.081194       9 inference_session_encoder_util.cc:81] Creating sessions for 3 pipelines...
I0000 00:00:1689266394.081203       9 inference_session_encoder_util.cc:85] Creating session for pipeline 'decoder_init_0'...
I0000 00:00:1689266394.083106       9 inference_session_encoder_util.cc:85] Creating session for pipeline 'decoder_step_embed_ids_0'...
I0000 00:00:1689266394.083430       9 inference_session_encoder_util.cc:85] Creating session for pipeline 'decoder_step_0'...
I0000 00:00:1689266394.086410       9 inference_session_encoder_util.cc:81] Creating sessions for 4 pipelines...
I0000 00:00:1689266394.086441       9 inference_session_encoder_util.cc:85] Creating session for pipeline 'encoder_embed_ids_0'...
I0000 00:00:1689266394.086873       9 inference_session_encoder_util.cc:85] Creating session for pipeline 'encoder_swizzled_birnn_0'...
I0000 00:00:1689266394.088002       9 inference_session_encoder_util.cc:85] Creating session for pipeline 'encoder_swizzled_birnn_1'...
I0000 00:00:1689266394.089084       9 inference_session_encoder_util.cc:85] Creating session for pipeline 'encoder_swizzled_birnn_2'...
I0000 00:00:1689266394.089595       9 decoder.cc:112] Creating decoder
I0000 00:00:1689266394.089610       9 inference_session_encoder_util.cc:81] Creating sessions for 3 pipelines...
I0000 00:00:1689266394.089617       9 inference_session_encoder_util.cc:85] Creating session for pipeline 'decoder_init_0'...
I0000 00:00:1689266394.091105       9 inference_session_encoder_util.cc:85] Creating session for pipeline 'decoder_step_embed_ids_0'...
I0000 00:00:1689266394.091398       9 inference_session_encoder_util.cc:85] Creating session for pipeline 'decoder_step_0'...
I0000 00:00:1689266394.093753       9 extended_preprocessor.cc:118] Do in-memory preprocessing
W0000 00:00:1689266394.094577       9 preproc_add_period.cc:100] Option 'skip_operation' has been renamed 'skip_in_training'
I0000 00:00:1689266394.177090       9 extended_preprocessor.cc:118] Do in-memory preprocessing
W0000 00:00:1689266394.177572       9 preproc_add_period.cc:100] Option 'skip_operation' has been renamed 'skip_in_training'
I0000 00:00:1689266394.220560       9 wordpiece_preprocessor.cc:46] Model loaded from pb string. Verifying symbols.
I0000 00:00:1689266394.221534       9 wordpiece_preprocessor.cc:81] Symbols=5998 chars=401
I0000 00:00:1689266394.304579       9 extended_preprocessor.cc:118] Do in-memory preprocessing
W0000 00:00:1689266394.305058       9 preproc_add_period.cc:100] Option 'skip_operation' has been renamed 'skip_in_training'
I0000 00:00:1689266394.347433       9 wordpiece_preprocessor.cc:46] Model loaded from pb string. Verifying symbols.
I0000 00:00:1689266394.348318       9 wordpiece_preprocessor.cc:81] Symbols=5998 chars=401
I0000 00:00:1689266394.380747       9 wordpiece_preprocessor.cc:46] Model loaded from pb string. Verifying symbols.
I0000 00:00:1689266394.381594       9 wordpiece_preprocessor.cc:81] Symbols=5998 chars=401
I0000 00:00:1689266394.407096       9 wordpiece_preprocessor.cc:46] Model loaded from pb string. Verifying symbols.
I0000 00:00:1689266394.407691       9 wordpiece_preprocessor.cc:81] Symbols=3998 chars=401
I0000 00:00:1689266394.411029       9 dictionary.cc:266] Dictionary build date: 20191024
MyApp(34414,0x30c81f000) malloc: *** error for object 0x30c81de28: pointer being freed was not allocated
MyApp(34414,0x30c81f000) malloc: *** set a breakpoint in malloc_error_break to debug
(lldb)

malloc_error_break breakpoint trace:

libsystem_malloc.dylib`malloc_error_break:
->  0x1110ba885 <+0>:  pushq  %rbp
    0x1110ba886 <+1>:  movq   %rsp, %rbp
    0x1110ba889 <+4>:  nop    
    0x1110ba88a <+5>:  nopl   (%rax)
    0x1110ba88e <+9>:  popq   %rbp
    0x1110ba88f <+10>: retq