[FIREBASE_STORAGE]: java.lang.OutOfMemoryError after repeated use of putData()/putFile()

[CloudMountain]

Is there an existing issue for this?

• [X] I have searched the existing issues.

Which plugins are affected?

Storage

Which platforms are affected?

Android

Description

Running what is essentially the following:

Uint8List buffer = Uint8List(5000000);   
Reference ref =  FirebaseStorage.instance.ref();   
UploadTask? uploadTask = await ref.child(path/to/file.pcm).putData(buffer);

seems to leave behind a ByteBuffer of length 5,000,000, and a second ByteBuffer of length ~4,190,000 each time it is called. The source of these two according to the Android Studio profiler are as follows:

I have an application which would expect to upload ~300 5MB chunks over the course of a session and am seeing OOM of the following kind: Fatal Exception: java.lang.OutOfMemoryError: Failed to allocate a 2576 byte allocation with 1020864 free bytes and 996KB until OOM, target footprint 268435456, growth limit 268435456; giving up on allocation because <1% of heap free after GC. for some users after around 40 calls of the putData function.

Note: I have also tried writing the data to a file and using putFile instead of putData, but I see exactly the same behaviour.

I am also getting the following warning every time an upload finishes successfully: W/StorageTask(11365): unable to change internal state to: INTERNAL_STATE_CANCELED, INTERNAL_STATE_CANCELING isUser: true from state:INTERNAL_STATE_SUCCESS which seems to imply that after the upload has been successful, a cancel function is being called, but that cancel function is unable to complete.

Reproducing the issue

Create a button which has the following onPressed

Uint8List buffer = Uint8List(5000000);
Reference ref =  FirebaseStorage.instance.ref();
UploadTask? uploadTask = await ref.child(path/to/file.pcm).putData(buffer);

Each time it is tapped data is allocated to the java heap that is never deallocated.

Firebase Core version

3.2.0

Flutter Version

3.22.2

Relevant Log Output

D/UploadTask(11365): Waiting 0 milliseconds
W/StorageUtil(11365): Error getting App Check token; using placeholder token instead. Error: com.google.firebase.FirebaseException: No AppCheckProvider installed.
D/UploadTask(11365): Increasing chunk size to 524288
D/UploadTask(11365): Waiting 0 milliseconds
W/StorageUtil(11365): Error getting App Check token; using placeholder token instead. Error: com.google.firebase.FirebaseException: No AppCheckProvider installed.
D/UploadTask(11365): Increasing chunk size to 1048576
D/UploadTask(11365): Waiting 0 milliseconds
W/StorageUtil(11365): Error getting App Check token; using placeholder token instead. Error: com.google.firebase.FirebaseException: No AppCheckProvider installed.
D/UploadTask(11365): Increasing chunk size to 2097152
D/UploadTask(11365): Waiting 0 milliseconds
W/StorageUtil(11365): Error getting App Check token; using placeholder token instead. Error: com.google.firebase.FirebaseException: No AppCheckProvider installed.
D/UploadTask(11365): Increasing chunk size to 4194304
D/UploadTask(11365): Waiting 0 milliseconds
W/StorageUtil(11365): Error getting App Check token; using placeholder token instead. Error: com.google.firebase.FirebaseException: No AppCheckProvider installed.
W/StorageTask(11365): unable to change internal state to: INTERNAL_STATE_CANCELED, INTERNAL_STATE_CANCELING isUser: true from state:INTERNAL_STATE_SUCCESS

Flutter dependencies

Expand Flutter dependencies snippet

```yaml Dart SDK 3.4.3 Flutter SDK 3.22.2 dependencies: - android_intent_plus 5.1.0 [flutter platform meta] - async 2.11.0 [collection meta] - audio_session 0.1.21 [flutter flutter_web_plugins rxdart meta] - cloud_firestore 5.2.1 [cloud_firestore_platform_interface cloud_firestore_web collection firebase_core firebase_core_platform_interface flutter meta] - community_material_icon 5.9.55 [flutter] - connectivity_plus 5.0.2 [flutter flutter_web_plugins connectivity_plus_platform_interface js meta nm] - convert 3.1.1 [typed_data] - cupertino_icons 1.0.8 - firebase_analytics 11.2.1 [firebase_analytics_platform_interface firebase_analytics_web firebase_core firebase_core_platform_interface flutter] - firebase_auth 5.1.4 [firebase_auth_platform_interface firebase_auth_web firebase_core firebase_core_platform_interface flutter meta] - firebase_core 3.3.0 [firebase_core_platform_interface firebase_core_web flutter meta] - firebase_crashlytics 4.0.4 [firebase_core firebase_core_platform_interface firebase_crashlytics_platform_interface flutter stack_trace] - firebase_storage 12.1.3 [firebase_core firebase_core_platform_interface firebase_storage_platform_interface firebase_storage_web flutter] - flutter 0.0.0 [characters collection material_color_utilities meta vector_math sky_engine] - flutter_blue_plus 1.14.11 [flutter] - flutter_hooks 0.20.5 [flutter] - flutter_localizations 0.0.0 [flutter intl characters clock collection material_color_utilities meta path vector_math] - flutter_sound 9.2.13 [path_provider recase uuid provider path synchronized logger flutter flutter_sound_platform_interface flutter_sound_web] - flutter_sound_platform_interface 9.2.13 [flutter meta plugin_platform_interface logger] - flutter_svg 2.0.10+1 [flutter http vector_graphics vector_graphics_codec vector_graphics_compiler] - flutter_widget_from_html_core 0.15.1 [csslib flutter html logging] - hooks_riverpod 2.5.1 [collection flutter flutter_hooks flutter_riverpod riverpod state_notifier] - html 0.15.4 [csslib source_span] - html_unescape 2.0.0 - http 1.2.2 [async http_parser meta web] - internet_connection_checker_plus 2.2.0 [connectivity_plus flutter http] - logger 1.4.0 - path_provider 2.1.4 [flutter path_provider_android path_provider_foundation path_provider_linux path_provider_platform_interface path_provider_windows] - permission_handler 11.3.1 [flutter meta permission_handler_android permission_handler_apple permission_handler_html permission_handler_windows permission_handler_platform_interface] - sanitize_html 2.1.0 [html meta] - shared_preferences 2.3.1 [flutter shared_preferences_android shared_preferences_foundation shared_preferences_linux shared_preferences_platform_interface shared_preferences_web shared_preferences_windows] - stop_watch_timer 3.2.0 [rxdart] - url_launcher 6.3.0 [flutter url_launcher_android url_launcher_ios url_launcher_linux url_launcher_macos url_launcher_platform_interface url_launcher_web url_launcher_windows] - uuid 3.0.7 [crypto] - wakelock_plus 1.2.7 [flutter flutter_web_plugins meta wakelock_plus_platform_interface win32 dbus package_info_plus web] - webview_flutter 4.8.0 [flutter webview_flutter_android webview_flutter_platform_interface webview_flutter_wkwebview] dev dependencies: - flutter_launcher_icons 0.13.1 [args checked_yaml cli_util image json_annotation path yaml] - flutter_lints 4.0.0 [lints] - flutter_test 0.0.0 [flutter test_api matcher path fake_async clock stack_trace vector_math leak_tracker_flutter_testing async boolean_selector characters collection leak_tracker leak_tracker_testing material_color_utilities meta source_span stream_channel string_scanner term_glyph vm_service] - mockito 5.4.4 [analyzer build code_builder collection dart_style matcher meta path source_gen test_api] transitive dependencies: - _fe_analyzer_shared 67.0.0 [meta] - _flutterfire_internals 1.3.40 [collection firebase_core firebase_core_platform_interface flutter meta] - analyzer 6.4.1 [_fe_analyzer_shared collection convert crypto glob meta package_config path pub_semver source_span watcher yaml] - archive 3.6.1 [crypto path] - args 2.5.0 - boolean_selector 2.1.1 [source_span string_scanner] - build 2.4.1 [analyzer async convert crypto glob logging meta package_config path] - built_collection 5.1.1 - built_value 8.9.2 [built_collection collection fixnum meta] - characters 1.3.0 - checked_yaml 2.0.3 [json_annotation source_span yaml] - cli_util 0.4.1 [meta path] - clock 1.1.1 - cloud_firestore_platform_interface 6.3.1 [_flutterfire_internals collection firebase_core flutter meta plugin_platform_interface] - cloud_firestore_web 4.1.1 [_flutterfire_internals cloud_firestore_platform_interface collection firebase_core firebase_core_web flutter flutter_web_plugins] - code_builder 4.10.0 [built_collection built_value collection matcher meta] - collection 1.18.0 - connectivity_plus_platform_interface 1.2.4 [flutter meta plugin_platform_interface] - crypto 3.0.3 [typed_data] - csslib 1.0.0 [source_span] - dart_style 2.3.6 [analyzer args collection path pub_semver source_span] - dbus 0.7.10 [args ffi meta xml] - fake_async 1.3.1 [clock collection] - ffi 2.1.2 - file 7.0.0 [meta path] - firebase_analytics_platform_interface 4.2.1 [_flutterfire_internals firebase_core flutter meta plugin_platform_interface] - firebase_analytics_web 0.5.9+1 [_flutterfire_internals firebase_analytics_platform_interface firebase_core firebase_core_web flutter flutter_web_plugins] - firebase_auth_platform_interface 7.4.3 [_flutterfire_internals collection firebase_core flutter meta plugin_platform_interface] - firebase_auth_web 5.12.5 [firebase_auth_platform_interface firebase_core firebase_core_web flutter flutter_web_plugins http_parser meta web] - firebase_core_platform_interface 5.2.0 [collection flutter flutter_test meta plugin_platform_interface] - firebase_core_web 2.17.4 [firebase_core_platform_interface flutter flutter_web_plugins meta web] - firebase_crashlytics_platform_interface 3.6.40 [_flutterfire_internals collection firebase_core flutter meta plugin_platform_interface] - firebase_storage_platform_interface 5.1.27 [_flutterfire_internals collection firebase_core flutter meta plugin_platform_interface] - firebase_storage_web 3.9.12 [_flutterfire_internals async firebase_core firebase_core_web firebase_storage_platform_interface flutter flutter_web_plugins http meta web] - fixnum 1.1.0 - flutter_riverpod 2.5.1 [collection flutter meta riverpod state_notifier] - flutter_sound_web 9.2.13 [flutter_sound_platform_interface flutter logger flutter_web_plugins meta js] - flutter_web_plugins 0.0.0 [flutter characters collection material_color_utilities meta vector_math] - glob 2.1.2 [async collection file path string_scanner] - http_parser 4.0.2 [collection source_span string_scanner typed_data] - image 4.2.0 [archive meta xml] - intl 0.19.0 [clock meta path] - js 0.6.7 [meta] - json_annotation 4.9.0 [meta] - leak_tracker 10.0.4 [clock collection meta path vm_service] - leak_tracker_flutter_testing 3.0.3 [flutter leak_tracker leak_tracker_testing matcher meta] - leak_tracker_testing 3.0.1 [leak_tracker matcher meta] - lints 4.0.0 - logging 1.2.0 - matcher 0.12.16+1 [async meta stack_trace term_glyph test_api] - material_color_utilities 0.8.0 [collection] - meta 1.12.0 - nested 1.0.0 [flutter] - nm 0.5.0 [dbus] - package_config 2.1.0 [path] - package_info_plus 8.0.2 [ffi flutter flutter_web_plugins http meta path package_info_plus_platform_interface web win32 clock] - package_info_plus_platform_interface 3.0.1 [flutter meta plugin_platform_interface] - path 1.9.0 - path_parsing 1.0.1 [vector_math meta] - path_provider_android 2.2.9 [flutter path_provider_platform_interface] - path_provider_foundation 2.4.0 [flutter path_provider_platform_interface] - path_provider_linux 2.2.1 [ffi flutter path path_provider_platform_interface xdg_directories] - path_provider_platform_interface 2.1.2 [flutter platform plugin_platform_interface] - path_provider_windows 2.3.0 [ffi flutter path path_provider_platform_interface] - permission_handler_android 12.0.8 [flutter permission_handler_platform_interface] - permission_handler_apple 9.4.5 [flutter permission_handler_platform_interface] - permission_handler_html 0.1.2 [flutter flutter_web_plugins permission_handler_platform_interface web] - permission_handler_platform_interface 4.2.2 [flutter meta plugin_platform_interface] - permission_handler_windows 0.2.1 [flutter permission_handler_platform_interface] - petitparser 6.0.2 [meta] - platform 3.1.5 - plugin_platform_interface 2.1.8 [meta] - provider 6.1.2 [collection flutter nested] - pub_semver 2.1.4 [collection meta] - recase 4.1.0 - riverpod 2.5.1 [collection meta stack_trace state_notifier] - rxdart 0.28.0 - shared_preferences_android 2.3.1 [flutter shared_preferences_platform_interface] - shared_preferences_foundation 2.5.1 [flutter shared_preferences_platform_interface] - shared_preferences_linux 2.4.1 [file flutter path path_provider_linux path_provider_platform_interface shared_preferences_platform_interface] - shared_preferences_platform_interface 2.4.1 [flutter plugin_platform_interface] - shared_preferences_web 2.4.2 [flutter flutter_web_plugins shared_preferences_platform_interface web] - shared_preferences_windows 2.4.1 [file flutter path path_provider_platform_interface path_provider_windows shared_preferences_platform_interface] - sky_engine 0.0.99 - source_gen 1.5.0 [analyzer async build dart_style glob path source_span yaml] - source_span 1.10.0 [collection path term_glyph] - stack_trace 1.11.1 [path] - state_notifier 1.0.0 [meta] - stream_channel 2.1.2 [async] - string_scanner 1.2.0 [source_span] - synchronized 3.1.0+1 - term_glyph 1.2.1 - test_api 0.7.0 [async boolean_selector collection meta source_span stack_trace stream_channel string_scanner term_glyph] - typed_data 1.3.2 [collection] - url_launcher_android 6.3.8 [flutter url_launcher_platform_interface] - url_launcher_ios 6.3.1 [flutter url_launcher_platform_interface] - url_launcher_linux 3.2.0 [flutter url_launcher_platform_interface] - url_launcher_macos 3.2.0 [flutter url_launcher_platform_interface] - url_launcher_platform_interface 2.3.2 [flutter plugin_platform_interface] - url_launcher_web 2.3.3 [flutter flutter_web_plugins url_launcher_platform_interface web] - url_launcher_windows 3.1.2 [flutter url_launcher_platform_interface] - vector_graphics 1.1.11+1 [flutter http vector_graphics_codec] - vector_graphics_codec 1.1.11+1 - vector_graphics_compiler 1.1.11+1 [args meta path_parsing xml vector_graphics_codec path] - vector_math 2.1.4 - vm_service 14.2.1 - wakelock_plus_platform_interface 1.2.1 [flutter plugin_platform_interface meta] - watcher 1.1.0 [async path] - web 0.5.1 - webview_flutter_android 3.16.6 [flutter webview_flutter_platform_interface] - webview_flutter_platform_interface 2.10.0 [flutter meta plugin_platform_interface] - webview_flutter_wkwebview 3.14.0 [flutter path webview_flutter_platform_interface] - win32 5.5.3 [ffi] - xdg_directories 1.0.4 [meta path] - xml 6.5.0 [collection meta petitparser] - yaml 3.1.2 [collection source_span string_scanner] ```

Additional context and comments

No response

[SelaseKay]

Hi @CloudMountain , thanks for the report. I am able to reproduce this issue.

[SelaseKay]

Hi @CloudMountain , after further digging, I've discovered that this has to do with the native underlying sdk. Please raise an issue on the firebase-android-sdk repository.

[CloudMountain]

Hi @SelaseKay, thanks for the updates. I have raised an issue in the firebase-android-sdk repository here: https://github.com/firebase/firebase-android-sdk/issues/6310

[CloudMountain]

Hi, I have done some profiling on iOS and am seeing a similar phenomenon

Each call to the putData() function is allocating memory that is never freed (just to be clear in the example in the screenshot I called the upload function 4 times, waiting each time until the TaskState was TaskState.success). The stacktrace is not readable, but stepping through with the debugger I see an equivalence with Android in that the data is allocated during a call to readValueOfType(), i.e. when sending the data over from Flutter to the native side.

Just to make sure this isn't me making a basic mistake here could someone confirm; with regards to the sample code:

Uint8List buffer = Uint8List(5000000);
Reference ref =  FirebaseStorage.instance.ref();
UploadTask? uploadTask = ref.child(path/to/file.pcm).putData(buffer);

is there something I should be doing in terms of clean up once the uploadTask has completed? There is nothing I have been able to find in the documentation or the official example to imply that this is necessary (or even possible as a user of the package). I have tried (trivially) adding

uploadTask?.whenComplete(() => uploadTask = null);

but this has had no effect on the issue stated above.

Also, @SelaseKay would it be possible for you to let me know what it is exactly that you dug up when investigating the original issue to let you know that this was a problem with the sdk, so that I can relay that information to the appropriate people.

[CloudMountain]

Hi,

I have done some more investigation and think I understand better what is happening. Talking just about Android for the time being and specifically the file FlutterFirebaseStoragePlugin.java there is a Map called streamHandlers. Calling putData() from a flutter app will eventually call referencePutData(), which calls registerEventChannel(_, __), which calls registerEventChannel(_, __, ___) which calls streamHandlers.put(). What is put into the streamHandlers Map is a TaskStateChannelStreamHandler which contains an UploadTask. That UploadTask contains copies of the data to be uploaded as well as AdaptiveStreamBuffers (i.e. the buffer split into chunks for uploading); this is the data that is allocated but not deallocated as presented in the screenshots in the original post. These buffers persist indefinitely. As far as I can tell the only way to clean up these streamHandlers is to call the removeEventListeners() method, but this is only called in onDetachedFromEngine() and didReinitializeFirebaseCore() i.e. the only way to clean up is to close the app.

(Note: I have tried locally modifying the file such that removeEventListeners() is called before calling registerEventChannel(_,__,___), and this does seem to solve the memory issue, i.e. the references to those large buffers are deleted and the garbage collector is able to reclaim the memory with each subsequent call of putData(), although I appreciate this is (probably) an incorrect use of that method.)

On iOS the situation is similar in the file FLTFirebaseStoragePlugin.m there is a _streamHandlers Map. Calling putData() calls referencePutDataApp(), which calls setupTaskListeners() which adds FLTTaskStateChannelStreamHandler to the _streamHandlers Map which contains a task. When creating the task the buffer to be uploaded is malloced and then never freed, and the reference to that buffer in the task in the _streamHandlers Map is never cleaned up either. Again there is a cleanupWithCompletion method which is only ever called from detachFromEngineOnRegistrar() and didReinitializeFirebaseCore().

So, I'm not sure what the intended behaviour is, but it seems like either onSuccess/onFailure/onComplete should call cleanup routines on the uploadTask to clear the data which was attempted to be uploaded, OR, there needs to be a way to trigger that cleanup from the flutter side for users who intend to upload more data than what would be a reasonable expectation of RAM afforded to their app by the OS.

[CloudMountain]

Hi, I am unfortunately blocked until this issue is resolved. Has there been any investigation using the information provided in my two most recent responses? I think they demonstrate that this is unlikely to be a problem with the underlying sdk meaning the tag "blocked: firebase-sdk" is no longer relevant, and the most recent post potentially points towards a solution.

Also, the issue here: https://github.com/firebase/flutterfire/issues/13460, seems related. That user is calling putFile() multiple times in a row and encountering an OOM error, which other than the size of the files involved (and therefore the number of calls needed to run out of memory) is the same issue I am having here. Fwiw, I have tried using putFile() to upload files of ~100MB, which admittedly might not result in the same behaviour as uploading files ~1GB, however the analysis in my most recent post still holds; references to portions of the data that is being uploaded, for each subsequent upload, are kept indefinitely in the streamHandlers object, until you eventually run out of memory.

[SelaseKay]

Hi @CloudMountain , sorry for the late response. This is being looked into.

[arioul89]

Hello guys,

Any update on this ?

Is there any workaround ?

Regards

[russellwheatley]

I can take a look at this. Thank you for the investigation, @CloudMountain 🙏

[arioul89]

On our side, this problem happens on some Android devices, especially Xiaomi devices, while trying to upload videos to Firebase Storage.

Already tried to add :

android:largeHeap="true"

In AndroidManifest.xml

And also :

org.gradle.jvmargs=-Xmx4608m

on gradle.properties

No problem on iPhone.

[russellwheatley]

Hey @CloudMountain - I have created a PR to fix the out of memory exception on android: https://github.com/firebase/flutterfire/pull/13508

Could you please test this on your app and let me know if it resolves your issue? 🙏 Also - if anyone else wants to try it out, it's on this branch: https://github.com/firebase/flutterfire/tree/storage-13385

[CloudMountain]

Hi @russellwheatley, apologies for the delay I was away from my desk yesterday. I have this morning tried out the change and it does indeed resolve the issue on Android. Many thanks!

Whilst I noticed this issue initially on Android (I was only seeing crashes on Android) hence the title, whilst doing some investigating I did note a similar phenomenon on iOS as mentioned in some of the comments above. Whilst I haven't seen any crashes on iOS yet, the memory appears to be mismanaged in the same way, meaning there is likely a point where the same OOM issue will happen on iOS too. Assuming that is reproducible on your end, should this issue be reopened to deal also with iOS? Alternatively, I could raise a new issue if that is preferable.

[russellwheatley]

@CloudMountain - Could you create a new issue? Thanks for the confirmation 🙏