Logs processing improvements

[pipex]

Release notes

• The supervisor no longer uses the database to store the last log sent timestamp. Using the database for this is not justified since journal logs are lost between reboots in most cases and even when they are not, the logs throughput is limited by the network speed and the storage provided by the backend, so overall the number of lost logs will be minimal.
• Removes use of the JSONStream module for log processing, switching to the native JSON.parse.
• It modifies the logs processing pipe to use an async Node pipeline, this will avoid memory leaks due to an error on the pipe. Because it uses async processing, it prevents the stream handling from overwhelming the event loop and ensures control is yielded back to the event loop while messages are being sent.
• This PR also modifies the balena log backend (what uploads logs to the API), to use async functions, also yielding control back to the event loop between messages and creating backpressure on the log stream.

Overall the changes above should result in less memory and less CPU being used when processing logs, ensuring proper operation of the supervisor even under high load scenarios

[alexgg]

@pipex not really comfortable with dropping logs as a workaround to a bug we cannot debug - could we raise it in a brainstorm call please?

[pipex]

@alexgg The benefits of the previous implementation are questionable, logs could still be dropped because of backpressure when sending to the backend, or you could end up with repeated logs because the timestamps are only flushed to db every 10 minutes. Without the database at most you'd be losing a few seconds of logs if the user containers start before the supervisor.

I don't see this as a workaround, is just a more resource conscious implementation

I'll bring it to a brainstorm

[pipex]

I self-certify!