search

k-int / gokb-phase1

Original GOKb repo - Moving to https://github.com/openlibraryenvironment/gokb
http://www.gokb.org
Other
11 stars 5 forks source link

TIPPs #554

Closed jhsolomon closed 7 years ago

jhsolomon commented 8 years ago

Working as expected. Do we think the number is accurate?

Contain: TIPP Persistent Id Title Status Package Platform

jhsolomon commented 8 years ago

@sosguthorpe there are currently 18226 TIPPS in Test and 98138 in Live. That seems like a big discrepancy. Can you check to see if that is the correct number?

sosguthorpe commented 8 years ago

@jhsolomon Looking at the logs I think there might be an issue with the import. I'm looking into this now but it looks like it has failed to complete so that would explain the difference.

jhsolomon commented 8 years ago

@kristenwilson Please retest later this week.

jhsolomon commented 7 years ago

Today (3 November) there are 37266 TIPPs. Perhaps the import is still timing out?

jhsolomon commented 7 years ago

I am still not sure the TIPPs are loading correctly. There are 37424 on Test vs 98138 on Live.

https://gokb-test.openlibraryfoundation.org/gokb/search/index?qbe=g%3A3tipps

http://gokb.openlibraryfoundation.org/gokb/search/index?qbe=g%3A3tipps

Is there a way to check on this? I know that with cleaning up the duplicate titles there will be fewer TIPPs than on Live.

sosguthorpe commented 7 years ago

I would expect that any TIPPs related to deleted packages not to be present on the test platform. If the number of packages is reduced, compared to live, because they are not current, then I think we expect that there would be less TIPPs also.

jhsolomon commented 7 years ago

I do not see any TIPPs on the search TIPPs page.

image

jhsolomon commented 7 years ago

TIPPS now present

jhsolomon commented 7 years ago

There are 31812 TIPPs today 12/12.

I will keep this open as a reminder to check the number of TIPPs on the next test and compare.

jhsolomon commented 7 years ago

same number of TIPPs 1/10/17

jhsolomon commented 7 years ago

The number of TIPPS has now increased to 31867 after ingesting 3 packages.

image

After ingesting a 4th package (Nature Publishing Group: Journals: 20170117) it went up to 31994. The number of titles has not increased, which is good, but it doesn't seem right that the TIPPs continue to increase, since these are updates to existing packages.

image

Please let me know if I am not understanding this correctly!

sosguthorpe commented 7 years ago

This is probably the expected behaviour. I fixed an issue with the ingest process being completely broken yesterday so the ingests you tested probably will exhibit strange behaviour now that it's fixed, but this should be the first update only. I just reingested "Nature Publishing Group: Journals: 20170117" and that hasn't increased the TIPP count at all.

jhsolomon commented 7 years ago

I just ingested Annual Reviews and the TIPPs number did go up again to 32053, which is 59 more than before (and the number of titles in the package). It seems like the TIPPs increase only happens the first time that the packages is loaded on Test. When I've ingested the same package again, the number has stayed the same for me, as well (e.g. Brepols or Brill).

Since this does not seem to be affecting the number of Titles or creating duplicates, that is probably ok. But I am still curious as to where the additional TIPPs are going. Will there be any downstream affect by creating these "extra" TIPPs?

sosguthorpe commented 7 years ago

And all the information for the TIPP i.e. the platform remains the same? If the platform changes it will mean a new TIPP. It might be that the count is taking into account the "deleted" TIPPs not yet removed from
the system? As you say it isn't actually changing the titles in the system or the links between the packages and the TIPPs themselves. It's one to keep an eye on, but don't think it to break the system. Happy to leave this open while we continue to test.

On 11/01/17 15:21, jhsolomon wrote:

I just ingested Annual Reviews and the TIPPs number did go up again to 32053, which is 59 more than before (and the number of titles in the package). It seems like the TIPPs increase only happens the first time that the packages is loaded on Test. When I've ingested the same package again, the number has stayed the same for me, as well (e.g. Brepols or Brill).

Since this does not seem to be affecting the number of Titles or creating duplicates, that is probably ok. But I am still curious as to where the additional TIPPs are going. Will there be any downstream affect by creating these "extra" TIPPs?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/k-int/gokb-phase1/issues/554#issuecomment-271896356, or mute the thread https://github.com/notifications/unsubscribe-auth/ACbSO7mifxTZc3rdiVl_O-iYJhQODRtsks5rRPN_gaJpZM4J-ZNX.

jhsolomon commented 7 years ago

Yes, the title, package, and platform have all remained the same. For example, in the Annual Review package:

Test Annual Review of Animal Biosciences gokb:TIPP:25931313:30323553 https://gokb-test.openlibraryfoundation.org/gokb/resource/show/org.gokb.cred.TitleInstancePackagePlatform%3A30649873

Live Annual Review of Animal Biosciences gokb:TIPP:28506:291054 http://gokb.openlibraryfoundation.org/gokb/resource/show/org.gokb.cred.TitleInstancePackagePlatform%3A291055

I guess a better question to ask is, since identifiers are different in Test than they were in Live, it would make sense to have all new TIPPs. But where do the "old" TIPPs go? Are they removed during Housekeeping?

I'll keep testing today and see if I come up with anything else. Thanks!

On Wed, Jan 11, 2017 at 12:17 PM, sosguthorpe notifications@github.com wrote:

And all the information for the TIPP i.e. the platform remains the same? If the platform changes it will mean a new TIPP. It might be that the count is taking into account the "deleted" TIPPs not yet removed from the system? As you say it isn't actually changing the titles in the system or the links between the packages and the TIPPs themselves. It's one to keep an eye on, but don't think it to break the system. Happy to leave this open while we continue to test.

On 11/01/17 15:21, jhsolomon wrote:

I just ingested Annual Reviews and the TIPPs number did go up again to 32053, which is 59 more than before (and the number of titles in the package). It seems like the TIPPs increase only happens the first time that the packages is loaded on Test. When I've ingested the same package again, the number has stayed the same for me, as well (e.g. Brepols or Brill).

Since this does not seem to be affecting the number of Titles or creating duplicates, that is probably ok. But I am still curious as to where the additional TIPPs are going. Will there be any downstream affect by creating these "extra" TIPPs?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/k-int/gokb-phase1/issues/554#issuecomment-271896356,

or mute the thread https://github.com/notifications/unsubscribe- auth/ACbSO7mifxTZc3rdiVl_O-iYJhQODRtsks5rRPN_gaJpZM4J-ZNX.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/k-int/gokb-phase1/issues/554#issuecomment-271931185, or mute the thread https://github.com/notifications/unsubscribe-auth/AMwAJUEgtLww2WU72Md3pREMju8tWnT7ks5rRQ7CgaJpZM4J-ZNX .

-- Jennifer Solomon GOKb Editor, Acquisitions and Discovery North Carolina State University Libraries 919-515-2743 j kristen_wilson@ncsu.eduhsolomo@ncsu.edu

jhsolomon commented 7 years ago

I think it's ok to close it again.