Crash due to "too many opened files" on Android 8.0.0

While downloading (in sequence) a lot of files (700+ in my case), a native crash occurs as the maximum simultaneous opened files limit is reached (error: Too many open files) Limit seems to be 1024 files open.

Does your download manager closes FS open connection properly ? Any guess?

Thanks

Which platform(s) does your issue occur on?

Android only version 8.0.0 (not reproducted on 9.0+)

Please, provide the following version numbers that your issue occurs with:

CLI: 6.5.0
Cross-platform modules: 6.5.1
Runtime(s): 6.5.0

Please, tell us how to recreate the issue in as much detail as possible.

Do a huge loop of urls (2000 to be sure, can be the same file saved locally in 2000 different local filenames) and download it

Is there any code involved?

download service:

@Injectable({
providedIn: 'root',
})
export class DownLoaderService {
public database: any;
downloader: Downloader = new Downloader();
constructor(
    private connectivityService: ConnectivityService,
    private translate: TranslateService
) {
    Downloader.init();
}
async downloadFile(apiUrl, filename, downloadlocation): Promise<DownloadEventData> {
    if (!this.connectivityService.isOnline()) {
        throw new Error(this.translate.instant('GENERAL.ERROR.NO_NETWORK'));
    }
    const headerHttp = { }
    const url = apiUrl;
    const zipDownloaderId = this.downloader.createDownload({
        url: url,
        headers: headerHttp,
        path: downloadlocation,
        fileName: filename,
    });
    return this.downloader
        .start(zipDownloaderId, (progressData: ProgressEventData) => { })
        .catch(error => {
            throw new Error(this.translate.instant('GENERAL.ERROR.DOWNLOAD_FILE'));
        })
        .then(data => {
            if (!fs.File.exists(data.path)) {
                throw new Error(this.translate.instant('GENERAL.ERROR.LOCAL_SAVE'));
            }
            console.log(`File downloaded successfully in ${data.path}`);
            return data;
        });
}
}

Used with :

const medias = [
    { serverFilePath : 'https://domain.com/file1.png' },
    // ... + 1000 others
];
let i = 0;
for (let media of medias) {
    if (i % 50 === 0) {
        console.log('cooldown');
        await new Promise(resolve => setTimeout(resolve, 2000));
    }
    const localMediaFilename = getRelativeLocalPathFromServerUrl(media.serverFilePath, LOCAL_ROOT_FOLDER);
    const localMediaFullPath = getFullLocalPathFromServerUrl(media.serverFilePath, LOCAL_ROOT_FOLDER);

    // check if media exists, and download it if needed
    if (!fs.File.exists(localMediaFullPath)) {
        // download if not already
        await this.downloaderService.downloadFile(media.serverFilePath, localMediaFilename, localPath).catch(err => {
            console.error(err);
        });
    }
    i++;
};

Error in logcat :

2020-07-09 11:24:45.093 13549-13549/com.example.myapp E/JS: Error handler Failed downloading lesson media file for maneuverId=203761 and url https://www.domain.com/file_15937643525850_1902931.jpg Error: Echec du téléchargement du fichier. Error: Echec du téléchargement du fichier. at new ZoneAwareError (file:///data/data/com.example.myapp/files/app/vendor.js:98946:33) at file:///data/data/com.example.myapp/files/app/bundle.js:4090:31 at ZoneDelegate.push.../node_modules/@nativescript/angular/zone-js/dist/zone-nativescript.js.ZoneDelegate.invoke (file:///data/data/com.example.myapp/files/app/vendor.js:98036:26) at Object.onInvoke (file:///data/data/com.example.myapp/files/app/vendor.js:70379:33) at ZoneDelegate.push.../node_modules/@nativescript/angular/zone-js/dist/zone-nativescript.js.ZoneDelegate.invoke (file:///data/data/com.example.myapp/files/app/vendor.js:98035:32) at Zone.push.../node_modules/@nativescript/angular/zone-js/di... 2020-07-09 11:24:45.142 13549-13650/com.example.myapp E/Parcel: fcntl(F_DUPFD_CLOEXEC) failed in Parcel::read, i is 0, fds[i] is -1, fd_count is 1, error: Too many open files 2020-07-09 11:24:45.142 13549-13650/com.example.myapp I/OpenGLRenderer: Checking abandoned damageFrame 2020-07-09 11:24:45.142 13549-13650/com.example.myapp I/OpenGLRenderer: Surface query width is 1200 2020-07-09 11:24:45.142 13549-13650/com.example.myapp A/OpenGLRenderer: Failed to set damage region on surface 0xc46eee98, error=EGL_BAD_ACCESS 2020-07-09 11:24:45.143 13549-13650/com.example.myapp A/libc: ../../buildtools/third_party/libc++abi/trunk/src/abort_message.cpp:72: abort_message: assertion "terminate_handler unexpectedly threw an exception" failed 2020-07-09 11:24:45.196 13549-15782/com.example.myapp E/NativeCrypto: AppData::create pipe(2) failed: Too many open files 2020-07-09 11:24:45.201 13549-13650/com.example.myapp I/chatty: uid=10291(com.example.myapp) RenderThread identical 247 lines 2020-07-09 11:24:45.201 13549-13650/com.example.myapp A/libc: ../../buildtools/third_party/libc++abi/trunk/src/abort_message.cpp:72: abort_message: assertion "terminate_handler unexpectedly threw an exception" failed 2020-07-09 11:24:45.201 13549-15782/com.example.myapp E/AndroidRuntime: FATAL EXCEPTION: OkHttp Dispatcher Process: com.example.myapp, PID: 13549 java.lang.RuntimeException: javax.net.ssl.SSLException: Unable to create application data at com.android.org.conscrypt.ConscryptFileDescriptorSocket.newSsl(ConscryptFileDescriptorSocket.java:163) at com.android.org.conscrypt.ConscryptFileDescriptorSocket.(ConscryptFileDescriptorSocket.java:154) at com.android.org.conscrypt.OpenSSLSocketFactoryImpl.createSocket(OpenSSLSocketFactoryImpl.java:149) at okhttp3.internal.connection.RealConnection.connectTls(RealConnection.java:291) at okhttp3.internal.connection.RealConnection.establishProtocol(RealConnection.java:270) at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:162) at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:257) at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:135) at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:114) at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:126) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:200) at okhttp3.RealCall$AsyncCall.execute(RealCall.java:147) at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1162) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:636) at java.lang.Thread.run(Thread.java:764) Caused by: javax.net.ssl.SSLException: Unable to create application data at com.android.org.conscrypt.NativeCrypto.SSL_new(Native Method) at com.android.org.conscrypt.SslWrapper.newInstance(SslWrapper.java:59)

Tried to isolate the bug to confirm it :

const arr = new Array<number>(2000).fill(0);
const date = Date.now();

const localRoot = fs.knownFolders.documents().path;
const localFolder = 'testperf';

const url = 'https://d1fmx1rbmqrxrr.cloudfront.net/cnet/optim/i/edit/2019/04/eso1644bsmall__w770.jpg';
for (let index in arr) {
    const loopUrl = url.replace('.jpg', '--' + date + '--' + index + '.jpg');
    const localFilename =  fs.path.join(localFolder, loopUrl.split('/').pop());
    const localFullPath = fs.path.join(localRoot, localFilename);

    if (!fs.File.exists(localFullPath)) {
        // see first post for the implementation of downloadService
        await this.downloaderService.downloadFile(url, localFilename, localRoot).catch(err => {
            console.error('err download:', err);
        });
    }
}

Running on a blank nativescript app on a Samsung Galaxy Tab A on Android 8.0.0. It starts catching errors at loop 388

And totally fails at index 788 with an error :

An uncaught Exception occurred on "Fancy Downloader" thread. pthread_create (1040KB stack) failed: Try again

StackTrace: java.lang.OutOfMemoryError: pthread_create (1040KB stack) failed: Try again at java.lang.Thread.nativeCreate(Native Method) at java.lang.Thread.start(Thread.java:733) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:970) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1388) at okhttp3.Dispatcher.enqueue(Dispatcher.java:132) at okhttp3.RealCall.enqueue(RealCall.java:100) at co.fitcom.fancydownloader.ManagerService$2.run(ManagerService.java:128) at android.os.Handler.handleCallback(Handler.java:790) at android.os.Handler.dispatchMessage(Handler.java:99) at android.os.Looper.loop(Looper.java:164) at android.os.HandlerThread.run(HandlerThread.java:65)

Another try, another crash (at loop 720) :

2020-07-17 14:09:40.533 28793-30754/org.nativescript.suivimobilens E/NativeCrypto: AppData::create pipe(2) failed: Too many open files 2020-07-17 14:09:40.551 28793-30754/org.nativescript.suivimobilens E/AndroidRuntime: FATAL EXCEPTION: OkHttp Dispatcher Process: org.nativescript.suivimobilens, PID: 28793 java.lang.RuntimeException: javax.net.ssl.SSLException: Unable to create application data at com.android.org.conscrypt.ConscryptFileDescriptorSocket.newSsl(ConscryptFileDescriptorSocket.java:163) at com.android.org.conscrypt.ConscryptFileDescriptorSocket.(ConscryptFileDescriptorSocket.java:154) at com.android.org.conscrypt.OpenSSLSocketFactoryImpl.createSocket(OpenSSLSocketFactoryImpl.java:149) at okhttp3.internal.connection.RealConnection.connectTls(RealConnection.java:291) at okhttp3.internal.connection.RealConnection.establishProtocol(RealConnection.java:270) at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:162) at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:257) at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:135) at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:114) at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:126) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121) at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:200) at okhttp3.RealCall$AsyncCall.execute(RealCall.java:147) at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1162) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:636) at java.lang.Thread.run(Thread.java:764)

Seems related to java handlers memory leak

triniwiz / nativescript-downloader