Another Batch of New Reports Ready.

[csaundrs]

These reports are ready for final review:

Acquisitions -> Funds and Ledgers -> list_all_ledger_ids.report Acquisitions -> Funds and Ledgers -> list_all_location_ids.report Acquisitions -> Funds and Ledgers -> list_all_fund_codes_and_names.report Acquisitions -> Purchase Orders and Line Items -> acq_stat_1_purchase_order_report.report Description of a Library's Collection -> General -> elink_bibs.report Description of a Library's Collection -> General -> elink_mfhds.report

[gibsonjc]

The results of these reports match the Access versions. Ready to go to prod!

[patrickzurek]

All have been moved to production. I usually run them again on prod just to double check (we've had an instance where a report ran on dev fine but not on prod).

There's kind of a problem with Description of a Library's Collection -> General -> elink_bibs.report Description of a Library's Collection -> General -> elink_mfhds.report

When I run those reports for DPU they generate a huge amount of output (~70MB csv file). Loading a result for viewing takes a relatively long while. I hazard its long enough that the user might think there's something wrong and try clicking the result link again to re-open it which will cause everything to start processing and downloading over again, especially if they're accustomed to the normally speedy loading times of other results.

Paging works better, but it's still a little sluggish. Downloading CSV takes a little bit of time too before the file is ready to be saved and the download to begin.

I checked and DPU has an enrollment of ~5,000 students (did not check the actual library collection stats). So that puts them in the small-medium range maybe? Can we expect this amount of output for these two reports from most other institutions too then?

It also causes significant CPU to be used for the duration of when the report is loading: but since it's unlikely that many other users will be viewing the results for this same report at the same time I'm not as concerned.

So what can we do? Add a note to the report template page that loading the results for this particular report may take longer than normal?

Also, do we need to consider the load impact these reports might have on the production oracle server?

[gibsonjc]

Thanks for doing extra testing, Pat. This report is going to be very large for large institutions because it is basically reporting on every URL they have in their bibliographic/holdings records.

These concerns your raised that these may pose problems for CPU, server load, and/or page loading times are serious, so I think we should remove them from production for now and let's discuss these things and do more testing on devel. No one is expecting these two reports to be there, so let's not tempt fate until we understand all of the issues better.

[gibsonjc]

...so I'm proposing that you take down: Description of a Library's Collection -> General -> elink_bibs.report Description of a Library's Collection -> General -> elink_mfhds.report from production for now.

[patrickzurek]

I just ran 'Elink Bibs' for UIU and it took maybe 5-6 minutes or so (I didn't time it exactly). But I do know that it took 222 seconds of actual Oracle processing to run the report (there's other overhead time associated with running the report too). It generated an intermediate output file on the web-reports server 594MB in size.

Trying to load the report for viewing on the site is taking a very long time - in fact it probably hung (I don't see any CPU being consumed anymore, but the page still says loading) We may have to consider not making this report available.

[patrickzurek]

Ah, I was drafting my second note and I didn't even realize you responded.

I will remove them from prod.

[patrickzurek]

They've been removed from prod and remain on devel. I'll close this issue because the other new reports (presumably) do not have any problems on prod. I'll create a new issue for the two problem reports in question.