search

eXtensibleCatalog / test

Testing
MIT License
0 stars 0 forks source link

MST and Drupal summary stats reconcile #99

Closed patrickzurek closed 7 years ago

patrickzurek commented 7 years ago

JIRA issue created by: rcook Originally opened: 2012-04-25 03:24 PM

Issue body: (nt)

This issue has attachments associated with it (external link): [drupal stats summary.png](http://www.carli.illinois.edu/sites/files/XCfiles/drupal stats summary.png) [dt off.png](http://www.carli.illinois.edu/sites/files/XCfiles/dt off.png) [dt off2.png](http://www.carli.illinois.edu/sites/files/XCfiles/dt off2.png) [mst stats summary.png](http://www.carli.illinois.edu/sites/files/XCfiles/mst stats summary.png) [summary stats.png](http://www.carli.illinois.edu/sites/files/XCfiles/summary stats.png)

patrickzurek commented 7 years ago

JIRA Comment by user: rcook JIRA Timestamp: 2012-04-25 03:28 PM

Comment body:

Attached are some docs related to reconciling MST and Drupal in total. Fairly straight forward I think, but please improve. Please chime in on any missing data elements or things that would be helpful to know in addition to what is here. Note: the screen shots do not reconcile in numbers, they are for informative purposes.

patrickzurek commented 7 years ago

JIRA Comment by user: jbrand JIRA Timestamp: 2012-04-30 12:40 PM

Comment body:

This is assigned to me, but it is unclear to me what I am supposed to do next. Reviewing this, it looks good, as for output from DCToXCTransformation, it should produce a decent log with numbers showing that for each bib, get 1 work, 1 expression, 1 manifestation. This was fixed recently by me though so not released till 1.4 mst.

patrickzurek commented 7 years ago

JIRA Comment by user: rcook JIRA Timestamp: 2012-04-30 01:43 PM

Comment body:

This one I thought was pretty straight forward. I think the next steps are for Peter/MT to do a reconcile and prove this/document this (at least for ourselves. Moving to Drupal.

patrickzurek commented 7 years ago

JIRA Comment by user: rcook JIRA Timestamp: 2012-05-08 10:39 AM

Comment body:

Last week, when I looked at one harvest that Peter pointed me to, we were 11 records off btween MST and DT, but these appeared to be from the before I looked, so I don't know where the difference entered. and were in the holdings, and all M. W adm E matched. So the one harvest I compared validated. Now we are 73 records diff in W and E, 13 diff in M and 28 in H. I will attach the screen shots.

This is requires close, daily monitoring, so that the second a discrepancy is found, it can be researched.

I will add screen shots.

patrickzurek commented 7 years ago

JIRA Comment by user: rcook JIRA Timestamp: 2012-05-08 10:42 AM

Comment body:

These are snaps of the logs, that show sometime between last week and now the numbers don't tie out.

patrickzurek commented 7 years ago

JIRA Comment by user: rcook JIRA Timestamp: 2012-05-08 10:55 AM

Comment body:

[~admin] I think we have lost the value in the large test system that has been set up and running. See this issue. What is status of case RCL-13. I have a feeling, a start over is needed, probably many more needed, and we can't keep taking days of processing time to nail this down.

patrickzurek commented 7 years ago

JIRA Comment by user: rcook JIRA Timestamp: 2012-05-08 03:57 PM

Comment body:

I tried to be helpful and try to diagnose when things got out of sync.

4/27 - Initial harvest. Numbers match except 11 too few records in DT.
4/29 - First small harvest. Numbers reconcile aside from the 11 holding records that were a problem from the start. 4/30 - Numbers reconcile aside from the 11 holding records that were a problem from the start. then you didn't do regular harvests. 5/3 - is the next harvest. Some deletes have happened in the intervening time and the DT log issue I reported in drupal-466 I think prevents me from telling any more details.

patrickzurek commented 7 years ago

JIRA Comment by user: rcook JIRA Timestamp: 2012-05-21 03:13 PM

Comment body:

MT, this is an issue related to your question this morning about the MST logs.

patrickzurek commented 7 years ago

JIRA Comment by user: mwesley JIRA Timestamp: 2012-07-30 04:38 AM

Comment body:

Moving this to: Harvested records match MST at #DRUPAL-98 -- from months of testing, I've concluded that the problem is with the harvested deletes of manifestation records not being deleted in Drupal after the harvest -- although they are clearly marked for deletion. A fix is in the works.

patrickzurek commented 7 years ago

Issue resolved: 2012-07-30 04:38 AM