When the user wants to export his/her logs, the system needs to combine multiple log files. Also, since on iOS we have multiple processes (foreground application & background process), we are generating multiple log files simultaneously (i.e. log files with overlapping timestamps).
The previous implementation was quite inefficient. It would:
parse every logFile into an in-memory Array
combine all these arrays into a giant array
sort the giant array by timestamp
concatenate all the log entries into a giant string
convert the giant string into UTF-8 data
before finally writing the data to a tempFile for export
In addition to being quite inefficient, the biggest problem here is the memory usage. If there are 10 MB of log data, we might use 30-40 MB of memory to perform these operations. Which, on older devices, may result in the process being killed by the OS.
This PR uses an AsyncChannel to stream data from the LogFileParser to the exporter. Importantly, back pressure is used to ensure the reading process doesn't get ahead of the writing/export process, which ensures that memory pressure is kept to a minimum during the export process.
When the user wants to export his/her logs, the system needs to combine multiple log files. Also, since on iOS we have multiple processes (foreground application & background process), we are generating multiple log files simultaneously (i.e. log files with overlapping timestamps).
The previous implementation was quite inefficient. It would:
In addition to being quite inefficient, the biggest problem here is the memory usage. If there are 10 MB of log data, we might use 30-40 MB of memory to perform these operations. Which, on older devices, may result in the process being killed by the OS.
This PR uses an AsyncChannel to stream data from the LogFileParser to the exporter. Importantly, back pressure is used to ensure the reading process doesn't get ahead of the writing/export process, which ensures that memory pressure is kept to a minimum during the export process.