Exporting Performance Woes and Inconsistencies

[LePips]

When exporting as Text/HTML/PDF, the app remains frozen while the underlying data is being compiled/rendered on the main thread. The performance is quite poor for small stores and gets much worse as they get larger.

Testing

Store sizes are given as indicated under Store Details.

For a 7MB store (11 Pro Max), with Start Date removed to export all logs:

	Text	HTML	PDF
Size	-	-	-
Time	crash	cancelled, too long/warm device	cancelled, too long/warm device

For a 1.2MB store (11 Pro Max), exporting the current session via Sharing Options:

	Text	HTML	PDF
Size	5.2MB	35MB	n/a
Time	~5s	~15s	n/a

For a 172KB store (11 Pro Max), using the Pulse Demo iOS app with Start Date removed to export all logs:

	Text	HTML	PDF
Size	324KB	1.7MB	584MB
Time	~1s	>1s	~40s

If you think that testing was done incorrectly, I am welcome for instructions and to retry.

Inconsistencies

I only compare inconsistencies of exports as plain text for simplification.

• Share as Text orders logs in descending order of time, Share Store -> Output Format: Plain Text orders in ascending order (as expected)
• Share as Text does not have a name for the file, Share Store -> Output Format: Plain Text properly names the file with a timestamp

Since there are two places to export, the Share as X menu and the Sharing Options modal, it can be confusing for which route to use for getting a proper list of logs. I also think it can be redundant since the Sharing Options modal allows specifying a time range and output format for the entire store. It would probably be best to aggregate sharing the store into Sharing Options which already has filtering, format selection and background rendering/cancellation.

While it is understood through usage that the Share as X menu refers to the current list it can just be a bit confusing, especially when I will include instructions in my application for users to share their logs.

For individual logs the existing sharing menu is just fine.

Other

• Share Store -> Output Format: Plain Text has a period at the end of the file name, resulting in a double period with the file type. ex: logs-2023-01-11-14-49..txt

[kean]

Hi, @LePips. Thanks for trying it out and I appreciate the detailed report. I'm currently working on 3.0.1 and it's a perfect time to fix these defects, including the performance ones. I had no time to measure it. I'm sure there are some low-handing fruits that would improve this drastically.

PDF export is pretty slow regardless, so it might be worth disabling it completely for store sharing. I tested rendering 70 pages and it was slow.

rendered on the main thread

Should be relatively easy to move it to the background and provide the user progress. It could display the number of processed messages.

Plain text formatting currently goes through the following flow: entity -> NSAttributedString -> plain text. The intermediate NSAttributedString step could be removed entirely if necessary for performance. But it should decided based on measurements.

can be confusing for which route to use for getting a proper list of logs

Yeah, there are probably too many places for sharing right now, but they are all useful in their own way.

• In Console, "Share As" shares currently displayed entities (uses "sharing" style for tasks). It shares whatever is in the list, which means you can shared filtered logs. This is why it's the primary sharing option for the screen (and a new feature).
• In Console context menu, "Share Store" shares the entire store by default and supports a couple of simple filters. I kind of want to get rid of the filters. They are redundant, yes. The fastest option is to share the entire store as .pulse file because it requires no processing - only copy. I think it's pretty clear what is does.
• In Text Mode, "Share As" shares whatever text is rendered on the screen with the selected filters/styles/expanded messages. This is probably the most flexible one, but also potentially the most confusing. Basically whenever you see share with a text view – it shares what you see.

include instructions in my application for users to share their logs

I would definitely not expose this menu to the users. If you want to provide them with a simple option, add a custom button to your app and use LoggerStore.copy(to:) to create a shareable archive (.pulse). You'll have to use Pulse Pro to view it, but you'll have all the data and you can easily filter it when you receive it.

does not have a name for the file

Seems like a bug, as well as the extra period. Thanks for the report, it is very much appreciated.

[kean]

I optimized the entity -> NSAttributedString path and it's now 7x faster and there are a couple more potential areas of improvement. It was also a significant (~40%) portion of the entity -> NSAttributedString -> HTML path so it is also now faster.

There are a couple of low-hanging fruits in NSAttributedString -> HTML path as well, but I haven't yet addressed them. And PDF will just be slow. I'm considering only enabling it for single-request shares.

Even with these improvements, sharing massive amounts of messages will still take a nontrivial amount of time. So I'm definitely going to be moving this work to the background and allowing the user to cancel it. I want to add a progress bar as well where possible.

One area I haven't explored yet is parallelization. It's going to be trivial to parallelize the entity -> NSAttributedString path. For HTML and PDF – not sure yet. But it could bring another massive performance boost.

[kean]

Please test with 3.1.0. I'm sure there are still some improvements that could be made, so I'm open to ideas.