psu-libraries / scholarsphere-3

A web application for ingest, curation, search, and display of digital assets. Powered by Hydra technologies (Rails, Hydra-head, Blacklight, Solr, Fedora Commons, etc.)
Apache License 2.0
78 stars 24 forks source link

Analytics page is throwing an error on every record #934

Closed carolyncole closed 7 years ago

carolyncole commented 7 years ago

As described by Jeffrey Edmunds" jhe2@psu.edu the analytics page is just erroring

To reproduce this pick any work of files set on scholarsphere.psu.edu and click on the analytics button

carolyncole commented 7 years ago

The page is not showing correct information, but at least it is not showing an error after I updated the configs for ticket #929

It looks like for migrated files the migration date is showing not the deposited date

mtribone commented 7 years ago

I've also found that the @stats.created_date in the labeling at the top is having an issue with string to date.

https://github.com/samvera/sufia/blob/f189645d84bc2c796df444ed0b85d8ca84bc84c2/app/presenters/sufia/stats_usage_presenter.rb#L29

mtribone commented 7 years ago

Screen Shot 2017-06-14 at 8.39.26 AM.png

carolyncole commented 7 years ago

https://gist.github.com/cam156/613af195d8a8a7b55d6559db84a229db

carolyncole commented 7 years ago

ref #842

GenericFile View should translate into a work View GenericFile Download should translate into a FileSet Download.

The user should be able to see total views on a work, total views for a file set, and total downloads for a file set.

carolyncole commented 7 years ago

The Data is in the wrong table and will not be displayed.

Any Items in the workview table prior to june 8th, 2017 are bogus and should be removed

We need to move all FileViews prior to June 8th, 2017 into the WorkView table transforming views column to work_views column, and the file_id column to the work_id column.

carolyncole commented 7 years ago

Saved the original tables

create table work_view_stats_sav as select * from work_view_stats;
create table file_view_stats_sav as select * from file_view_stats;

tested 1 file in the mysql console with these commands:

delete from work_view_stats where work_id = 'p8418n32c' and date < '2017-06-08';
insert into work_view_stats select * from file_view_stats where file_id = 'p8418n32c' and date < '2017-06-08';
carolyncole commented 7 years ago
delete  from file_view_stats where length(file_id) = 10 and date < '2017-06-08';
delete from work_view_stats where date < '2017-06-08';
insert into work_view_stats (date, work_views, work_id, created_at, updated_at, user_id) select date,views,file_id, created_at, updated_at, user_id from file_view_stats where date < '2017-06-08';
carolyncole commented 7 years ago

Commands to revert the table

drop table work_view_stats
create table work_view_stats as select * from work_view_stats_sav;
create index index_work_view_stats_on_user_id on work_view_stats (user_id) using btree;
create index index_work_view_stats_on_work_id on work_view_stats (work_id) using btree;
awead commented 7 years ago

Update the FileViewStat table with the file set ids from each work:

ActiveFedora::SolrService.query("has_model_ssim:GenericWork", rows: 10_000).each do |hit|
  if hit["member_ids_ssim"].present? && hit["member_ids_ssim"].count == 1
    FileViewStat.where(file_id: hit["id"]).update_all(file_id: hit["member_ids_ssim"].first)
    FileDownloadStat.where(file_id: hit["id"]).update_all(file_id: hit["member_ids_ssim"].first)
  end
end
carolyncole commented 7 years ago

I ran the code above on production this morning. The only open issue is to run the deployment on Production tomorrow morning.