search

k-int / gokb-phase1

Original GOKb repo - Moving to https://github.com/openlibraryenvironment/gokb
http://www.gokb.org
Other
11 stars 5 forks source link

Warning messages for start/end dates in Refine and Review Tasks in GOKb #484

Closed jhsolomon closed 8 years ago

jhsolomon commented 8 years ago

This is related to #480

I tested this package: Elsevier: ScienceDirect Backfile Package - Agricultural and Biological Sciences 1995-2004: 20150122

In Refine I used the automatic date adjustment, but I still created about 25 date review tasks in GOKb.

We discussed during the call on March 18 that there may be some relation to Refine changing the date from 2005 to 2005-03-18, and that causing a conflict if the GOKb date is 2005-01-01. However, I could not replicate this today.

jhsolomon commented 8 years ago

fix confirmed. @kristenwilson please test.

kristenwilson commented 8 years ago

I'm not seeing that this is fixed. Here's a title I'm looking at in OpenRefine with a coverage start date of 1962: start_date_error_1

And here's that same title in GOKb, with a publication start date of 1970. start_date_error_2

This situation should be creating a warning message (which you can see it's not), since the start date of the coverage pre-dates the publication start date.

Jennifer, if you have a functioning counter-example that caused you to believe that this issue is fixed, can you please share it? We need to figure out if this is a consistent problem or only occurring in certain situations.

jhsolomon commented 8 years ago

One example is "Progress in Lipid Research" in Elsevier: ScienceDirect Backfile Package - Biochemistry Genetics And Molecular Biology Legacy. I can look for some additional examples if that would be helpful.

Here is the title with the warning in Refine:

image

The TI in GOKb:

image

Then Refine after using the autocorrect function:

image

jhsolomon commented 8 years ago

But then this package created 9 review tasks because of TIPP end dates. So there is something not working correctly.

image

ianibo commented 8 years ago

Going to add more information to the request text - can't see whats going wrong at the moment. Is there a review request above that relates to the row for Progress in lipid research?

jhsolomon commented 8 years ago

This time, there were 4 titles with incorrect pre-dates and these were resolved with the automatic adjustment.

image

But for the post-date, all 114 titles were flagged incorrectly. And the auto adjustment did not work. The warning message was still there after trying it twice.

image

jhsolomon commented 8 years ago

From the first ingest of this package, no, there were no review tasks for Progress in lipid Research.

ianibo commented 8 years ago

@sosguthorpe Can you call me first thing thursday about this? need to talk it through with you :)

ianibo commented 8 years ago

(I think it's the case that there are 9 errors in this file, so 9 of the titles should be flagged as true for the post-date check. The review request is flagging these, but refine is missing them for some reason)

ianibo commented 8 years ago

Guys - Steve and I would like to do a test - Can you please update one of the titles in a package and make its publishedTo date -48h after the date in the package you are ingesting, similarly adjust a second one and make it +48h, then re-run the ingest and see if we get any different results? ty.

jhsolomon commented 8 years ago

Pre-ingest date changes: image

After ingesting, there were 16 date review tasks, but these did not include the two titles I changed: Advances in Biophysics and Advances In Enzyme Regulation.

jhsolomon commented 8 years ago

I then went back and used the auto correct function and reingested the package. This time there was only 1 review task for a missing TIPP.

image

jhsolomon commented 8 years ago

sorry was looking at the wrong title. The missing TIPP "Mutation Research/Fundamental and Molecular Mechanisms of Mutagenesis" is in the package. It should have been ingested.

jhsolomon commented 8 years ago

Regarding TIPP dates: I ingested Karger: Hospital Collection: 20160420 and then added the Published from dates for two titles: Acta Cytologica and Advances in Psychosomatic Medicine to GOKb Test.

I then went back to Refine and changed the firstdateissue to 1950 for both titles re-ingested and got 2 review tasks.

Just to make sure that I understand, we should not see any review warnings for pre or post dates in Refine because this feature is currently turned off?

ianibbo commented 8 years ago

I think it's [Review requests] turned on at the moment - should I switch it off?

I think refine should also be detecting these before ingest tho - did it not?

Ian Ibbotson Director Knowledge Integration Ltd 35 Paradise Street, Sheffield. S3 8PZ T: 0114 273 8271 M: 07968 794 630 W: http://www.k-int.com Doodle: http://doodle.com/ianibbo

On 20 April 2016 at 20:48, jhsolomon notifications@github.com wrote:

Regarding TIPP dates: I ingested Karger: Hospital Collection: 20160420 and then added the Published from dates for two titles: Acta Cytologica and Advances in Psychosomatic Medicine to GOKb Test.

I then went back to Refine and changed the firstdateissue to 1950 for both titles re-ingested and got 2 review tasks.

Just to make sure that I understand, we should not see any review warnings for pre or post dates in Refine because this feature is currently turned off?

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/k-int/gokb-phase1/issues/484#issuecomment-212576824

jhsolomon commented 8 years ago

No, there were no warnings in Refine. I thought you did switch these off last week. Or Steve did?

On Wed, Apr 20, 2016 at 4:03 PM, ianibbo notifications@github.com wrote:

I think it's [Review requests] turned on at the moment - should I switch it off?

I think refine should also be detecting these before ingest tho - did it not?

Ian Ibbotson Director Knowledge Integration Ltd 35 Paradise Street, Sheffield. S3 8PZ T: 0114 273 8271 M: 07968 794 630 W: http://www.k-int.com Doodle: http://doodle.com/ianibbo

On 20 April 2016 at 20:48, jhsolomon notifications@github.com wrote:

Regarding TIPP dates: I ingested Karger: Hospital Collection: 20160420 and then added the Published from dates for two titles: Acta Cytologica and Advances in Psychosomatic Medicine to GOKb Test.

I then went back to Refine and changed the firstdateissue to 1950 for both titles re-ingested and got 2 review tasks.

Just to make sure that I understand, we should not see any review warnings for pre or post dates in Refine because this feature is currently turned off?

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/k-int/gokb-phase1/issues/484#issuecomment-212576824

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub https://github.com/k-int/gokb-phase1/issues/484#issuecomment-212581633

Jennifer Solomon GOKb Editor, Acquisitions and Discovery North Carolina State University Libraries 919-515-2743 j kristen_wilson@ncsu.eduhsolomo@ncsu.edu

ianibo commented 8 years ago

Sorry - yes - switched off in the currently deployed version

jhsolomon commented 8 years ago

I retested using: Taylor & Francis Sport, Leisure & Tourism 2016: test 1 Taylor & Francis Sport, Leisure & Tourism 2016: test 2

When I changed the dates in the file to pre-date and post-date the publication dates in GOKb, I did not receive any warnings in Refine. The review tasks correctly noted these errors in the CRED.

On Fri, Apr 22, 2016 at 4:38 AM, Ian Ibbotson notifications@github.com wrote:

Sorry - yes - switched off in the currently deployed version

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub https://github.com/k-int/gokb-phase1/issues/484#issuecomment-213327306

Jennifer Solomon GOKb Editor, Acquisitions and Discovery North Carolina State University Libraries 919-515-2743 j kristen_wilson@ncsu.eduhsolomo@ncsu.edu

jhsolomon commented 8 years ago

I am going to close this issue because the review tasks are working and #491 and #490 address the Refine warnings.