Related to issue #129, I am opening this issue to discuss an enhancement I'd like to propose. I suggest that we make changes to the Web-Reports backend to automatically generate the CSV output for all reports after a report is submitted to be run in Web-Reports.
The operation of generating the CSV does not happen asynchronously but rather synchronously (in other words, the generation is only triggered when the user clicks the download CSV link) so the user cannot navigate away from the page or do anything else in that browser's tab/window except wait for the download to start & complete (possibly several minutes or longer depending on the report).
Let's assume the user wants to download the CSV file for a report that will take several minutes to be generated. If the user does not notice in the status bar of their browser the message "Waiting for web-reports-devel.carli.illinois.edu..." AND also does not correctly associate the message with their act of clicking the download CSV file link, they may try clicking the link again thereby resetting the process or even just give up entirely.
My proposed enhancement is to automatically generate a CSV output file for every report submitted through Web-Reports. This makes the CSV generation an asynchronous operation that the user is not necessarily waiting on.
For this to be beneficial, the time devoted to generating the CSV file after a report is submitted will not be tied to the process of actually running the report on Oracle/making it available for viewing in the web interface. We don't want to delay the web interface view availability of all reports across the board for the subset of users who will want the CSV output.
Here's the pros and cons that I've come up with:
Pros:
When a user wants to download a CSV file, it is available immediately and they will get a file download notification or dialogue depending on their browser right after clicking the download link.
Will not require much work beyond that already required for #129. Some additional effort is required of course, but again I would categorize it as a small amount.
Cons:
An additional but negligible amount of disk space is required on the server. We have plenty available.
Additional processing resources on the server will be required every time a report is run by a user. If the report run would have had its CSV file downloaded eventually, this is not a wasted effort. If the CSV file will never be downloaded, the server's processing would have been wasteful. However, I am curious if there's a positive correlation between the length of time it takes a CSV file to be generated and the likelihood a user will want to download the output as CSV anyway. The reasoning being that reports with many thousands of rows are sometimes best handled as csv output instead of the browser view anyway.
To be further researched by me is whether this will delay the processing of any subsequent reports the user may submit after submitting a previous report. It should, but I'd like to look into the implications of this and how to minimize it.
To summarize, I propose that we automatically generate a CSV output file every time a user runs a report. It will negligibly impact server disk resources, but possibly lead to cases where the server has performed unnecessary work if the CSV file is never accessed. The big benefit is that for reports that take several minutes to generate, the file will be available for immediate download. This last thing is my primary motivation for proposing this. We don't have a good way of letting the user know that the server is working on generating their file behind the scenes (and I think, anything we could do to improve that aspect would require more effort than my proposal here) and so we're left with whatever message the user's browser displays to them about "waiting on the server..." Tech savvy users may pick up on this. Others less familiar may not be.
Related to issue #129, I am opening this issue to discuss an enhancement I'd like to propose. I suggest that we make changes to the Web-Reports backend to automatically generate the CSV output for all reports after a report is submitted to be run in Web-Reports.
The operation of generating the CSV does not happen asynchronously but rather synchronously (in other words, the generation is only triggered when the user clicks the download CSV link) so the user cannot navigate away from the page or do anything else in that browser's tab/window except wait for the download to start & complete (possibly several minutes or longer depending on the report).
Let's assume the user wants to download the CSV file for a report that will take several minutes to be generated. If the user does not notice in the status bar of their browser the message "Waiting for web-reports-devel.carli.illinois.edu..." AND also does not correctly associate the message with their act of clicking the download CSV file link, they may try clicking the link again thereby resetting the process or even just give up entirely.
My proposed enhancement is to automatically generate a CSV output file for every report submitted through Web-Reports. This makes the CSV generation an asynchronous operation that the user is not necessarily waiting on.
For this to be beneficial, the time devoted to generating the CSV file after a report is submitted will not be tied to the process of actually running the report on Oracle/making it available for viewing in the web interface. We don't want to delay the web interface view availability of all reports across the board for the subset of users who will want the CSV output.
Here's the pros and cons that I've come up with:
Pros:
Cons:
To be further researched by me is whether this will delay the processing of any subsequent reports the user may submit after submitting a previous report. It should, but I'd like to look into the implications of this and how to minimize it.
To summarize, I propose that we automatically generate a CSV output file every time a user runs a report. It will negligibly impact server disk resources, but possibly lead to cases where the server has performed unnecessary work if the CSV file is never accessed. The big benefit is that for reports that take several minutes to generate, the file will be available for immediate download. This last thing is my primary motivation for proposing this. We don't have a good way of letting the user know that the server is working on generating their file behind the scenes (and I think, anything we could do to improve that aspect would require more effort than my proposal here) and so we're left with whatever message the user's browser displays to them about "waiting on the server..." Tech savvy users may pick up on this. Others less familiar may not be.
@gibsonjc, @csaundrs, @dmcmpbll, @jenhm: Thoughts?