CUB-Libraries-CTA / counter-data-loader

Loads COUNTER database from JR1 report spreadsheets
1 stars 2 forks source link

Review MySQL Database Indexes #82

Open ericnienhouse opened 1 year ago

ericnienhouse commented 1 year ago

Estimate: 3

Long running queries are causing timeouts in the COUNTER app UI.

Acceptance Criteria:

Export/dump COUNTER database from "production like" environment. Identify dump file size. Identify MySQL server version. Determine suitable storage location. Ideally: Store on NCAR LAN accessible storage (glade, Stratus, etc).

Timebox to 2 days max.

ericnienhouse commented 1 year ago

Example query

SELECT t.title, t.publisher, p.preferred_name as platform, Max(m.period) as max_period, Min(m.period) as min_period, SUM(m.period_total) AS total_requests, m.title_report_id, t.isbn, t.yop, t.uri, t.doi FROM title_report t INNER JOIN platform_ref p on t.platform_id=p.id INNER JOIN metric m on t.id = m.title_report_id WHERE ( access_type IN (:access_type_1,:access_type_0) AND metric_type = :metric_type AND period<= :end_date AND period>= :start_date AND t.title_type= :title_type AND m.title_type= :title_type) GROUP BY t.title, t.publisher, p.preferred_name , m.title_report_id, t.isbn, t.yop, t.uri,t.doi ORDER BY total_requests DESC, title ASC

PARAMS: { 'title_type': 'B', 'metric_type': 'Total_Item_Requests', 'access_type_1': 'OA_Gold', 'access_type_0': 'Controlled', 'start_date': '2022-01-01', 'end_date': '2022-12-01' }

bonnland commented 1 year ago

Dump size: 3.6 GB Server version: 5.7.18 Metrics table size: ~ 31 million rows

Dump location: /glade/scratch/bonnland