itinance / react-native-fs

Native filesystem access for react-native
MIT License
4.94k stars 975 forks source link

Error when reading large file on Android #598

Open mannok opened 5 years ago

mannok commented 5 years ago

I am trying to read 2xxmb crypt12 file from FS. When I trying to run await RNFS.readFile(${RNFS.ExternalStorageDirectoryPath}/Whatsapp/Databases/msgstore.db.crypt12), it prompts up

Could not invoke RNFSManager.readFile

null

Failed to allocate a 536870928 byte allocation with 25165824 free bytes and 248MB until OOM, max allowed footprint 301365936, growth limit 536870912

I have already set android:hardwareAccelerated="false" and android:largeHeap="true" in AndroidManifest.xml but no luck.

How can I solve it? Thank you

itinance commented 5 years ago

Well, such things happen when you are blowing your devices memory with such a big file. It was never a good practice to load a file completely into memory. You should treat this instead on a chunk-by-chunk basis. Best thing here would be to find a library, that is capable of decrypting the crypt12-file. Then you can load this database (assuming it is Sqlite3 based) with react-native-sqlite-storage to dig trough the data.

There is little chance, that whatsapp is using sqlite-cipher for encyrption (DYOR pls). Then this issue should give you help: https://github.com/craftzdog/react-native-sqlite-2/issues/32

lyudad commented 4 years ago

same issue, I tried to upload 30 sec video

sej1228 commented 4 years ago

同样的问题,我尝试上传30秒的视频

Did you solve this problem later

same issue, I tried to upload 30 sec video

Did you solve this problem later

lyudad commented 4 years ago

I have tried rn-fetch-blob library instead and it work for me

sej1228 commented 4 years ago

I have tried rn-fetch-blob library instead and it work for me

RNFetchBlob.fs.readStream?

lyudad commented 4 years ago

no const data = RNFetchBlob.wrap(localPath) RNFetchBlob.fetch('POST', url, { 'Content-Type': 'multipart/form-data' }, [ { name: 'file', filename: 'test', data, type: this.mediaMimeType } ]) but I have found that it's slower than send data in base64. So I still looking for faster solution

shengliZh commented 4 years ago

Hi, Any update about this issue? I have the same issue.

RahulkrishnanSpericorn commented 4 years ago

Same issue

prafulla-codes commented 4 years ago

I am having the same issue any solution??

dolphinflow86 commented 3 years ago

Same here

SamiChab commented 3 years ago

Same here

aalhitennf commented 3 years ago

Getting same error. Device is fresh android emulator with android 11, lots of free space and 800mb free memory so that shouldn't be problem. Error doesn't happen everytime tho, with small files never, ~10mb file, sometimes/often, bigger than that, always.

AliRehman7141 commented 2 years ago

Any update on this issue?

Marbos21 commented 2 years ago

+1

ghost commented 1 year ago

+1

CrisRondaS commented 1 year ago

Any update on this issue?

adrienGzc commented 1 year ago

Same here

jjmayorgaq commented 1 year ago

Hello, any help with this error?

jim-jigx commented 1 year ago

Same issue here. I am using the read() method to chunk 5Mb segments of a 1Gb file and can process about 300Mb before this fails.

dzakki commented 8 months ago

same here. any update with this issue ?

AlexAJ9 commented 8 months ago

+1

favaztechcurve commented 7 months ago

+1

favaztechcurve commented 7 months ago

same here. any update with this issue ?

@dzakki did you find the issue?

dzakki commented 7 months ago

same here. any update with this issue ?

@dzakki did you find the issue?

I haven't identified the issue yet

7adidaz commented 5 months ago

still facing a similar issue when using RNFS.read with a reasonable chunk size, it looks like it tried to allocate memory larger than the chunk size requested.

saadlatif75 commented 1 month ago

Any update to resolve this issue? @mannok @itinance @dolphinflow86 @sej1228 @lyudad