Correct way to write large base64/utf8 files and read them later

[iocuydi]

Hi, I'm trying to write large lists into storage, upload them to a server, and parse them later in python. These lists will be thousands or even millions of elements long, resulting in files of 100+MB.

When I write them in utf8 format, they can be parsed successfully later on, but if they are longer than a few thousand elements (~30 MB), the app hangs when writing the file, and the file never appears at the path. If they are short, the process works fine.

When I write them in base64 format, they are successfully written very quickly. Even huge files near 1 gb are successfully written to storage with this format. However, when I try to parse them later on, they are always corrupted. I've tried everything from standard stuff like: https://kite.com/python/examples/3422/base64-decode-a-%60base64%60-file to online base64 decoder services, so I'm fairly certain I'm not decoding it wrong.

Does anyone have a suggestion for how to

1. Write large utf8 files so the app doesn't crash OR
2. Write base64 files correctly so that they can be decoded later

My react native code is as follows:

var list = [1,1,1,....1,2] // can be 100000+ elements
var list2 = [2,2,2,...1,2] // can be 100000+ elements
var json = JSON.stringify({x: list, y: list2})

var path = RNFS.ExternalDirectoryPath + '/test.json';
//alert(JSON.stringify({'x': xlist, 'y': ylist, 'z': zlist}))

// write the file
let format = 'utf8' //OR 'base64'
RNFS.writeFile(path, json, format)
.then(async (success) => {

})
.catch((err) => {
});

[sinewave440hz]

I'm having a similar issue when writing audio files. The promise on writeFile is fulfilled but no file is created and no error is caught. Pretty hopeless for debugging...

[ghost]

same problem

[alashuk-applicaster]

any updates? same problem