search

Fujio-Turner / sg-log-reader-demo

Parsing and Aggregating Sync Gateway Logs
https://fujio-turner.github.io/sg-log-reader-demo/
Apache License 2.0
5 stars 1 forks source link

Add Async to speed up processing of file #12

Closed Fujio-Turner closed 1 year ago

Fujio-Turner commented 1 year ago

The processing of the wsID LIST can be done by different threads to speed up the processing of a sg_log file.

Fujio-Turner commented 1 year ago

Cleanup some items to use less memory and asyncio helps to process a 100MB sg_info.log file with 18,900 WebSocket Ids from 7 minutes to 5 minutes.