RTF output files become too large

[rdstern]

@ChrisMarsh82 and @lloyddewit I had a message from a new user who has been doing some ambitious analyses.

This follows our recent WMO workshops for South East Asia. The task was to tidy daily data for over 100 stations, for rainfall and temperatures. With the small problems they found, they first wasn’t sure it would work in R-Instat. They also wanted to merge the station details with the daily data.

It is now all ok and there are over 2 million daily records

The question is as follows: “The log file(.rtf) is super large. Do you have a way to read it or reduce the size?”

I hope you agree that’s an interesting question?

[rdstern]

Since being told to put this issue here, there have been messages from @dannyparsons, and others in the e-mail exchange. I collect them here.

From Danny:

I noticed that even saving 1 or 2 graphs in the output window the file size was very large so I don't think 
it's just about having lots of outputs. I think this is a problem for small outputs too.

Word or rtf output is important because it's editable and easy for users to adapt for a report. 
HTML isn't as good for this, so happy with it as an option, but not as the default.

On Thu, 12 Jan 2023 at 12:08, patrick munyoki wrote: Roger,

As mentioned by Danny, if it's the output window, below is the issue.

The maximum rtf file size that word or wordpad can open is 512 MB. 
The graphics that R produces can at times be very large, making the rtf produced by the output window become unusable.

Initially, I thought users will just select the specific outputs that they want to save and therefore that size limit will probably not be an issue, but I have increasingly found users wanting to save the whole output window. I wonder whether it's because we don't usually mention this to them?

In the short term, we could strive to make it clear that users should be saving the outputs of interest either in; 
1. Separate output window tab
2. As a file through the new maximise window feature.
3. As an R object. All outputs can now be saved as objects in the data book. Later they can always use R-Instat to reproduce the files.

In the future, I suggest we implement saving of the output window as a html. Linux has held us back on this.

Patrick"

On Thu, Jan 12, 2023 at 1:27 PM Danny Parsons wrote:

If it's the output window then that makes more sense! I do remember this being an issue and we should see why the file comes out as being so large.

Sorry Stephen for clogging up emails - as you suggested this should move to an issue discussion.

On Thu, 12 Jan 2023 at 10:25, patrick munyoki wrote: Hi Roger,

I'm surprised that the user was able to save the log file as an rtf file.
We don't have that as an option. We only have 2 file format options, namely; .R and .txt.
Which version is he using? or could he be talking about the output window?

Regards,

Patrick

On Thu, Jan 12, 2023 at 1:17 PM Danny Parsons wrote:

The log file should just be a plain text file or a .R file and should be very small. It only contains code so it 
shouldn't be an rft. If this is how it's exporting by default we should change this.

Roger - you could suggest they re-save it with a .txt or .r file to reduce the size.

[lloyddewit]

The log file is saved to an R or txt file. I don't think this issue applies to the log file so I changed the title and removed the log_script window label.