search

nextcloud / android

📱 Nextcloud Android app
https://play.google.com/store/apps/details?id=com.nextcloud.client
GNU General Public License v2.0
4.06k stars 1.74k forks source link

OutOfMemory exception in Nextcloud app when using the nextcloud/Android-SingleSignOn library to upload large files #13222

Open dipshit-enterprises opened 1 week ago

dipshit-enterprises commented 1 week ago

⚠️ Before posting ⚠️

Steps to reproduce

using the nextcloud/Android-SingleSignOn library:

  1. initiate an HTTP PUT request for WebDav
  2. use an InputStream to feed the request body
  3. have the request body be around 300 MB or more, and quickly accessible

Expected behaviour

The Nextcloud Files app copies a fixed/limited number of bytes from the request body's buffer from my app over the AIDL interface, making sure the bytes hit my Nextcloud server before reading more so that it doesn't run out of memory.

Actual behaviour

The Nextcloud Files app instead reads as much of the buffer as possible from my app over the AIDL interface with no limits and without waiting for the bytes to hit my server. It quickly approaches the JVM limit within seconds and crashes, as my app is able to provide the data much faster than it can be uploaded to my server.

Android version

13

Device brand and model

samsung SM-G975F

Stock or custom OS?

Custom (explain in "additional information")

Nextcloud android app version

3.29.1

Nextcloud server version

28.0.7

Using a reverse proxy?

Yes

Android logs

2024-07-02 21:27:34.308 17483-17483 ShowErrorActivity        D  onPause() ending
2024-07-02 21:27:34.907 17483-17483 ShowErrorActivity        D  onStop() ending
2024-07-02 21:27:34.912 17483-17483 ShowErrorActivity        D  onSaveInstanceState(Bundle) starting
2024-07-02 21:27:44.055 15564-15564 AppCompatDelegate        D  Checking for metadata for AppLocalesMetadataHolderService : Service not found
2024-07-02 21:27:44.069 15564-15564 SsoGrantPe...onActivity  D  onCreate(Bundle) starting
2024-07-02 21:27:44.086 15564-15564 Compatibil...geReporter  D  Compat change id reported: 210923482; UID 10213; state: ENABLED
2024-07-02 21:27:44.093 15564-15564 PackageManager           W  Failure retrieving resources for (my app package): Drawable (my app package):mipmap/ic_launcher_round with resource ID #0x7f0d0001
2024-07-02 21:27:44.103 15564-15564 Compatibil...geReporter  D  Compat change id reported: 171228096; UID 10213; state: ENABLED
2024-07-02 21:27:44.112 15564-15564 Compatibil...geReporter  D  Compat change id reported: 237531167; UID 10213; state: DISABLED
2024-07-02 21:27:44.117 15564-18914 Parcel                   W  Expecting binder but got null!
2024-07-02 21:27:44.125 15564-15564 com.ownclo...onActivity  V  TOKEN-REQUEST: Calling Package: (my app package)
2024-07-02 21:27:44.125 15564-15564 com.ownclo...onActivity  V  TOKEN-REQUEST: App Name: (my app name)
2024-07-02 21:27:44.127 15564-15564 SsoGrantPe...onActivity  D  onStart() starting
2024-07-02 21:27:44.128 15564-15564 MainApp                  D  APP IN FOREGROUND
2024-07-02 21:27:44.130 15564-15564 SsoGrantPe...onActivity  D  onResume() starting
2024-07-02 21:27:44.132 15564-18914 Parcel                   W  Expecting binder but got null!
2024-07-02 21:27:44.154 15564-18914 OpenGLRenderer           E  Unable to match the desired swap behavior.
2024-07-02 21:27:44.155 15564-18926 Gralloc4                 I  mapper 4.x is not supported
2024-07-02 21:27:44.155 15564-18926 Gralloc3                 W  mapper 3.x is not supported
2024-07-02 21:27:44.158 15564-18926 gralloc                  I  Arm Module v1.0
2024-07-02 21:27:44.160 15564-18926 Gralloc4                 W  allocator 4.x is not supported
2024-07-02 21:27:44.160 15564-18926 Gralloc3                 W  allocator 3.x is not supported
2024-07-02 21:27:44.183 15564-18914 OpenGLRenderer           E  Unable to match the desired swap behavior.
2024-07-02 21:28:16.375 15564-18914 OpenGLRenderer           D  endAllActiveAnimators on 0x7c16799080 (RippleDrawable) with handle 0x7b267d5fb0
2024-07-02 21:28:16.389 15564-15564 SsoGrantPe...onActivity  D  onPause() ending
2024-07-02 21:28:16.477 15564-15564 SsoGrantPe...onActivity  D  onStop() ending
2024-07-02 21:28:16.479 15564-15564 SsoGrantPe...onActivity  D  onDestroy() ending
2024-07-02 21:28:17.091 15564-15564 MainApp                  D  APP IN BACKGROUND
2024-07-02 21:28:18.010 15564-15577 NetworkUtils             D  Searching known-servers store at /data/user/0/com.nextcloud.client/files/knownServers.bks
2024-07-02 21:28:18.011 15564-15577 com.ownclo...KeyManager  D  init(): Loading SharedPreferences named com.nextcloud.client.AdvancedX509KeyManager
2024-07-02 21:28:18.012 15564-15577 com.ownclo...KeyManager  D  init(): keychain aliases = []
2024-07-02 21:28:18.014 15564-15577 com.ownclo...KeyManager  D  init(): Loading SharedPreferences named com.nextcloud.client.AdvancedX509KeyManager
2024-07-02 21:28:18.014 15564-15577 com.ownclo...KeyManager  D  init(): keychain aliases = []
2024-07-02 21:28:18.015 15564-15577 OwnCloudClient #0        D  Creating OwnCloudClient
2024-07-02 21:28:18.015 15564-15577 AccountUtils             D  Restoring cookies for (my account name)
2024-07-02 21:28:18.020 15564-15577 AccountUtils             D  Restoring cookies for (my account name)
2024-07-02 21:28:18.025 15564-15577 OwnCloudClient #0        D  REQUEST PUT /remote.php/dav/files/(my account id)/testing/large%20file%201.bin
2024-07-02 21:28:18.027 15564-15577 AdvancedSs...ketFactory  D  Creating SSL Socket with remote (my instance):443, local null:0, params: org.apache.commons.httpclient.params.HttpConnectionParams@40f9276
2024-07-02 21:28:18.028 15564-15577 AdvancedSs...ketFactory  D   ... with connection timeout 60000 and socket timeout 60000
2024-07-02 21:28:18.031 15564-15577 TrafficStats             D  tagSocket(106) with statsTag=0xffffffff, statsUid=-1
2024-07-02 21:28:18.032 15564-15577 ServerNameIndicator      I  SSLSocket implementation: org.conscrypt.Java8EngineSocket
2024-07-02 21:28:18.032 15564-15577 ServerNameIndicator      I  SNI done, hostname: (my instance)
2024-07-02 21:28:24.035 15564-15572 extcloud.client          I  Clamp target GC heap from 293MB to 256MB
2024-07-02 21:28:30.329 15564-15577 extcloud.client          I  Starting a blocking GC Alloc
2024-07-02 21:28:30.329 15564-15577 extcloud.client          I  Starting a blocking GC Alloc
2024-07-02 21:28:30.337 15564-15577 extcloud.client          I  Alloc young concurrent copying GC freed 11(31KB) AllocSpace objects, 0(0B) LOS objects, 22% free, 197MB/256MB, paused 35us,21us total 8.037ms
2024-07-02 21:28:30.338 15564-15577 extcloud.client          I  Forcing collection of SoftReferences for 256MB allocation
2024-07-02 21:28:30.338 15564-15577 extcloud.client          I  Starting a blocking GC Alloc
2024-07-02 21:28:30.364 15564-15577 extcloud.client          I  Alloc concurrent copying GC freed 4857(185KB) AllocSpace objects, 1(64MB) LOS objects, 41% free, 133MB/229MB, paused 32us,28us total 26.727ms
2024-07-02 21:28:30.365 15564-15577 extcloud.client          W  Throwing OutOfMemoryError "Failed to allocate a 268435468 byte allocation with 100663296 free bytes and 122MB until OOM, target footprint 240472872, growth limit 268435456" (VmSize 15250988 kB)
2024-07-02 21:28:30.365 15564-15577 extcloud.client          I  Starting a blocking GC Alloc
2024-07-02 21:28:30.365 15564-15577 extcloud.client          I  Starting a blocking GC Alloc
2024-07-02 21:28:30.370 15564-15577 extcloud.client          I  Forcing collection of SoftReferences for 256MB allocation
2024-07-02 21:28:30.370 15564-15577 extcloud.client          I  Starting a blocking GC Alloc
2024-07-02 21:28:30.389 15564-15577 extcloud.client          I  Alloc concurrent copying GC freed 14(16KB) AllocSpace objects, 0(0B) LOS objects, 41% free, 133MB/229MB, paused 17us,20us total 18.910ms
2024-07-02 21:28:30.389 15564-15577 extcloud.client          W  Throwing OutOfMemoryError "Failed to allocate a 268435472 byte allocation with 100663296 free bytes and 122MB until OOM, target footprint 240472648, growth limit 268435456" (VmSize 15250988 kB)
2024-07-02 21:28:30.389 15564-15577 JavaBinder               E  *** Uncaught remote exception!  (Exceptions are not yet supported across processes.)
                                                                java.lang.OutOfMemoryError: Failed to allocate a 268435472 byte allocation with 100663296 free bytes and 122MB until OOM, target footprint 240472648, growth limit 268435456
                                                                    at java.util.Arrays.copyOf(Arrays.java:3670)
                                                                    at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:120)
                                                                    at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:95)
                                                                    at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:156)
                                                                    at org.apache.commons.httpclient.methods.InputStreamRequestEntity.bufferContent(InputStreamRequestEntity.java:137)
                                                                    at org.apache.commons.httpclient.methods.InputStreamRequestEntity.getContentLength(InputStreamRequestEntity.java:187)
                                                                    at org.apache.commons.httpclient.methods.EntityEnclosingMethod.getRequestContentLength(EntityEnclosingMethod.java:336)
                                                                    at org.apache.commons.httpclient.methods.EntityEnclosingMethod.addContentLengthRequestHeader(EntityEnclosingMethod.java:406)
                                                                    at org.apache.commons.httpclient.methods.EntityEnclosingMethod.addRequestHeaders(EntityEnclosingMethod.java:374)
                                                                    at org.apache.commons.httpclient.HttpMethodBase.writeRequestHeaders(HttpMethodBase.java:2177)
                                                                    at org.apache.commons.httpclient.HttpMethodBase.writeRequest(HttpMethodBase.java:2060)
                                                                    at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1096)
                                                                    at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398)
                                                                    at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
                                                                    at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
                                                                    at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
                                                                    at com.owncloud.android.lib.common.OwnCloudClient.executeMethod(OwnCloudClient.java:192)
                                                                    at com.nextcloud.android.sso.InputStreamBinder.processRequestV2(InputStreamBinder.java:423)
                                                                    at com.nextcloud.android.sso.InputStreamBinder.performNextcloudRequestAndBodyStreamV2(InputStreamBinder.java:113)
                                                                    at com.nextcloud.android.sso.aidl.IInputStreamService$Stub.onTransact(IInputStreamService.java:101)
                                                                    at android.os.Binder.execTransactInternal(Binder.java:1280)
                                                                    at android.os.Binder.execTransact(Binder.java:1244)
2024-07-02 21:28:30.390 15564-15577 AndroidRuntime           E  FATAL EXCEPTION: binder:15564_2
                                                                Process: com.nextcloud.client, PID: 15564
                                                                java.lang.OutOfMemoryError: Failed to allocate a 268435472 byte allocation with 100663296 free bytes and 122MB until OOM, target footprint 240472648, growth limit 268435456
                                                                    at java.util.Arrays.copyOf(Arrays.java:3670)
                                                                    at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:120)
                                                                    at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:95)
                                                                    at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:156)
                                                                    at org.apache.commons.httpclient.methods.InputStreamRequestEntity.bufferContent(InputStreamRequestEntity.java:137)
                                                                    at org.apache.commons.httpclient.methods.InputStreamRequestEntity.getContentLength(InputStreamRequestEntity.java:187)
                                                                    at org.apache.commons.httpclient.methods.EntityEnclosingMethod.getRequestContentLength(EntityEnclosingMethod.java:336)
                                                                    at org.apache.commons.httpclient.methods.EntityEnclosingMethod.addContentLengthRequestHeader(EntityEnclosingMethod.java:406)
                                                                    at org.apache.commons.httpclient.methods.EntityEnclosingMethod.addRequestHeaders(EntityEnclosingMethod.java:374)
                                                                    at org.apache.commons.httpclient.HttpMethodBase.writeRequestHeaders(HttpMethodBase.java:2177)
                                                                    at org.apache.commons.httpclient.HttpMethodBase.writeRequest(HttpMethodBase.java:2060)
                                                                    at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1096)
                                                                    at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398)
                                                                    at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
                                                                    at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
                                                                    at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
                                                                    at com.owncloud.android.lib.common.OwnCloudClient.executeMethod(OwnCloudClient.java:192)
                                                                    at com.nextcloud.android.sso.InputStreamBinder.processRequestV2(InputStreamBinder.java:423)
                                                                    at com.nextcloud.android.sso.InputStreamBinder.performNextcloudRequestAndBodyStreamV2(InputStreamBinder.java:113)
                                                                    at com.nextcloud.android.sso.aidl.IInputStreamService$Stub.onTransact(IInputStreamService.java:101)
                                                                    at android.os.Binder.execTransactInternal(Binder.java:1280)
                                                                    at android.os.Binder.execTransact(Binder.java:1244)
2024-07-02 21:28:30.411 17483-17483 ShowErrorActivity        D  onStart() starting
2024-07-02 21:28:30.412 17483-17483 ShowErrorActivity        D  onResume() starting
2024-07-02 21:28:30.429 17483-17552 OpenGLRenderer           E  Unable to match the desired swap behavior.

Server error logs

N/A

Additional information

Please note that despite using the library, the crash is within the Nextcloud Files app, and I believe this is a result of how the Nextcloud Files app handles the requests.

In the provided logcat, the file transfer made it to 268.57 MB before running out of memory.

Custom OS info: I am running Lineage OS 20, compiled by myself

dipshit-enterprises commented 6 days ago

I have investigated the issue and it seems to be caused by the apache commons http library allocating the entire request body if not given the content length.

I have a potential fix, I'll make a PR once I test it.