Closed hpiwowar closed 3 years ago
Actually I just realized how to get a slightly more comprehensive list:
0015-8209 2157-4650 1867-8610 1755-599X 2522-8897 2213-3917 1758-2562 2572-8288 2053-0153 6183-1638 1555-7855 1520-6785 1041-3499 0365-9631 0272-7528 1013-9877 0828-6914 1934-8800 0014-4398 2029-2317 0022-7153 0952-4592 1547-4127 0966-7377 0095-960X 0005-2744 0029-7976 0323-889X 1687-9996 0955-3681 1932-0166 2164-5285 0003-8946 0037-8968 0197-5323 0253-5904 0008-5286 0031-5621 0366-0796 2043-7323 1752-9859 2475-0255 2590-1443 0967-0653 1996-9481 1878-013X 2472-4718 0031-5710 2234-8867 1040-5712 0556-1892 0255-2922 2043-7331 0894-8631 2193-0872 1344-7661 1879-2871 0738-0232 1608-4799 0096-8013 2194-6019 0095-9502 0095-9510 1521-3979 2331-3439 1522-9580 2210-9110 0254-6272 1096-9136 2234-5957 2029-0187 2331-3420 1862-2720
All of these ISSNs have been imported into the database. If it still returns a 404 it is because there is no valid title or publisher data in the API response from either issn.org or crossref, like 1934-8800:
http://api.crossref.org/journals/1934-8800 https://portal.issn.org/resource/ISSN/1934-8800?format=json
I'm still working through making sure other data is available such as dois_by_year.
You might want to try WorldCat if it isn't in other places:
https://www.worldcat.org/search?q=1934-8800&qt=results_page#%2528x0%253Ajrnl%2Bx4%253Adigital%2529format
I don't know if they have an API.
We are going to want to find the publisher and title and any synonym ISSNs manually (via Googling) if we can't find them in the databases.
Guessing whether it is faster for you to do it or for you to assemble a list and train one of our admins to do it depends on how many there are? Can you update with best course of action as you figure it out? Thanks!
Working on it now! Worldcat's API seems to be restricted, but using wikidata seems promising:
Once you have the ID you can get the rest of the data fairly easily. I'm going to try it with some of the other ISSNs and see if this gives a good solution.
Good find! Yup, the fields we care most about are:
h
Heather Piwowar, cofounder
Our Research https://ourresearch.org/: tools to make scholarly research more open, connected, and reusable follow at @researchremix https://twitter.com/researchremix, @our_research https://twitter.com/our_research, @unpaywall, https://twitter.com/unpaywall and @unsub_org https://twitter.com/unsub_org
On Wed, May 19, 2021 at 9:56 AM caseydm @.***> wrote:
Working on it now! Worldcat's API seems to be restricted, but using wikidata seems promising:
Once you have the ID you can get the rest of the data fairly easily. I'm going to try it with some of the other ISSNs and see if this gives a good solution.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ourresearch/journalsdb/issues/16#issuecomment-844289894, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABWPAOH4ABUTGRC2ACQZUDTOPUUDANCNFSM447HS6WA .
so to be more explicit: we should do whatever it takes (manual data collection if necessary) to get those items.
Implicit in that is also issnl, which means we need to know is it actually a synonym of a journal we already have.
Heather
On Wed, May 19, 2021 at 11:22 AM Heather Piwowar @.***> wrote:
Good find! Yup, the fields we care most about are:
- title
- publisher
- is it currently publishing (dois in 2021 != 0)
- is it a gold OA journal (for now == is it on unpaywall's gold oa list)
- if it is currently publishing, subscription and apc data nice if we can get it
h
Heather Piwowar, cofounder
Our Research https://ourresearch.org/: tools to make scholarly research more open, connected, and reusable follow at @researchremix https://twitter.com/researchremix, @our_research https://twitter.com/our_research, @unpaywall, https://twitter.com/unpaywall and @unsub_org https://twitter.com/unsub_org
On Wed, May 19, 2021 at 9:56 AM caseydm @.***> wrote:
Working on it now! Worldcat's API seems to be restricted, but using wikidata seems promising:
Once you have the ID you can get the rest of the data fairly easily. I'm going to try it with some of the other ISSNs and see if this gives a good solution.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ourresearch/journalsdb/issues/16#issuecomment-844289894, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABWPAOH4ABUTGRC2ACQZUDTOPUUDANCNFSM447HS6WA .
Hi Heather. I've been working on this and the more I think about I think we need to use a manual admin process for the remaining ISSNs. What I'm finding is most of the ones left are 'suppressed' or 'incorrect identifier' issns that were once associated with a journal and are still out in the wild. So you have to use a combination of wikidata, worldcat, and google to find the similar journal and then link it to the actual journal as a synonym.
Here are some examples:
2210-9110: https://www.wikidata.org/wiki/Q26842423. Listed as incorrect identifier for issn-l 2210-9099. recommend add as synonym to that issn-l. 1096-9136: https://www.worldcat.org/search?q=1096-9136&qt=owc_search. Title is Diabetic Medicine and publisher is Wiley. Recommend add to existing journal with same name/publisher (https://api.journalsdb.org/journals/0742-3071) as synonym. 2234-5957: https://www.wikidata.org/wiki/Q15816293 / incorrect identifier for existing issn-l 1226-086X. Add as synonym. 1758-2562: google search leads to https://portal.issn.org/resource/ISSN/1758-2652 which is Journal of the International Aids Society / add as synonym for existing title https://api.journalsdb.org/journals/1758-2652 1520-6785: https://www.worldcat.org/search?qt=worldcat_org_all&q=1520-6785 / Nonprofit counsel : the Law of Tax-Exempt Organizations monthly. / add as synonym for title Nonprofit Counsel https://api.journalsdb.org/journals/0742-3497
It would be easy to export these ISSNs so the work can be done as an admin function. Someone can search for existing titles in journalsdb using this query: http://api.journalsdb.org/journals/search/?query=Journal%20of%20the%20International%20AIDS%20Society
I'm happy to do this if you want me to, it will take a 2 hours or so to get through these. But it will of course slow me down on some other things and will be recurring as we get additional ISSNs. Let me know what you think!
Hi Casey. Good detective work. Ok if 2 hours I think you should do it, and as you do it document the process and decision making enough so we can use the docs to train someone who isn't as familiar w the domain as you (now :) ) are to do it. Guessing that means itll take you 3 or 4 hours... still with it. Sound good?
h
On Wed, May 19, 2021, 3:24 PM caseydm, @.***> wrote:
Hi Heather. I've been working on this and the more I think about I think we need to use a manual admin process for the remaining ISSNs. What I'm finding is most of the ones left are 'suppressed' or 'incorrect identifier' issns that were once associated with a journal and are still out in the wild. So you have to use a combination of wikidata, worldcat, and google to find the similar journal and then link it to the actual journal as a synonym.
Here are some examples:
2210-9110: https://www.wikidata.org/wiki/Q26842423. Listed as incorrect identifier for issn-l 2210-9099. recommend add as synonym to that issn-l. 1096-9136: https://www.worldcat.org/search?q=1096-9136&qt=owc_search. Title is Diabetic Medicine and publisher is Wiley. Recommend add to existing journal with same name/publisher ( https://api.journalsdb.org/journals/0742-3071) as synonym. 2234-5957: https://www.wikidata.org/wiki/Q15816293 / incorrect identifier for existing issn-l 1226-086X. Add as synonym. 1758-2562: google search leads to https://portal.issn.org/resource/ISSN/1758-2652 which is Journal of the International Aids Society / add as synonym for existing title https://api.journalsdb.org/journals/1758-2652 1520-6785: https://www.worldcat.org/search?qt=worldcat_org_all&q=1520-6785 / Nonprofit counsel : the Law of Tax-Exempt Organizations monthly. / add as synonym for title Nonprofit Counsel https://api.journalsdb.org/journals/0742-3497
It would be easy to export these ISSNs so the work can be done as an admin function. Someone can search for existing titles in journalsdb using this query:
http://api.journalsdb.org/journals/search/?query=Journal%20of%20the%20International%20AIDS%20Society
I'm happy to do this if you want me to, it will take a 2 hours or so to get through these. But it will of course slow me down on some other things and will be recurring as we get additional ISSNs. Let me know what you think!
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ourresearch/journalsdb/issues/16#issuecomment-844537664, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABWPAJAOAFPDXDXUJFKO2DTOQ3ARANCNFSM447HS6WA .
Yes sounds good! I'll have these done early tomorrow and will document along the way.
Hi Heather. All of the updates are made so these ISSNs are pointing at the right journals. I am contacting Richard about some of the missing DOI and open access data. I'll also post the instructions I wrote a bit later. Here is a summary of what changed, with many of the ISSNs forwarding to existing journals:
Super, thanks Casey!
Heather Piwowar, cofounder
Our Research https://ourresearch.org/: tools to make scholarly research more open, connected, and reusable follow at @researchremix https://twitter.com/researchremix, @our_research https://twitter.com/our_research, @unpaywall, https://twitter.com/unpaywall and @unsub_org https://twitter.com/unsub_org
On Thu, May 20, 2021 at 11:47 AM caseydm @.***> wrote:
Hi Heather. All of the updates are made so these ISSNs are pointing at the right journals. I am contacting Richard about some of the missing DOI and open access data. I'll also post the instructions I wrote a bit later. Here is a summary of what changed, with many of the ISSNs forwarding to existing journals: ISSN Journals DB Endpoint Status Merge Into Missing Notes 0015-8209 https://api.journalsdb.org/journals/0015-8209 Good 2157-4650 https://api.journalsdb.org/journals/2157-4650 Fixed 1090-8471
1867-8610 https://api.journalsdb.org/journals/1867-8610 Good dois_by_year 1755-599X https://api.journalsdb.org/journals/1755-599X Good 2522-8897 https://api.journalsdb.org/journals/2522-8897 Good everything
2213-3917 https://api.journalsdb.org/journals/2213-3917 Good everything
1758-2562 https://api.journalsdb.org/journals/1758-2562 Fixed 1758-2652
2572-8288 https://api.journalsdb.org/journals/2572-8288 Good dois, open access 2053-0153 https://api.journalsdb.org/journals/2053-0153 Good, set publisher to Wiley Only on worldcat 6183-1638 https://api.journalsdb.org/journals/6183-1638 Fixed 0300-9084
1555-7855 https://api.journalsdb.org/journals/1555-7855 Good 1520-6785 https://api.journalsdb.org/journals/1520-6785 Fixed 0742-3497
1041-3499 https://api.journalsdb.org/journals/1041-3499 Fixed 1547-3368
0365-9631 https://api.journalsdb.org/journals/0365-9631 Good doi, open access 0272-7528 https://api.journalsdb.org/journals/0272-7528 Fixed 0148-0227 Wiley forwards to a different journal 'Journal of Geophysical Research' 1013-9877 https://api.journalsdb.org/journals/1013-9877 Fixed, set publisher 1838-8388 0828-6914 https://api.journalsdb.org/journals/0828-6914 Good doi, open access 1934-8800 https://api.journalsdb.org/journals/1934-8800 Fixed 0163-6383
0014-4398 https://api.journalsdb.org/journals/0014-4398 Good old journal
2029-2317 https://api.journalsdb.org/journals/2029-2317 Fixed 2029-882X German version of the journal 0022-7153 https://api.journalsdb.org/journals/0022-7153 Good 0952-4592 https://api.journalsdb.org/journals/0952-4592 Fixed 0016-8009
1547-4127 https://api.journalsdb.org/journals/1547-4127 Good 0966-7377 https://api.journalsdb.org/journals/0966-7377 Fixed 1476-993X
0095-960X https://api.journalsdb.org/journals/0095-960X Fixed 0022-3492
0005-2744 https://api.journalsdb.org/journals/0005-2744 Good 0029-7976 https://api.journalsdb.org/journals/0029-7976 Good old journal
0323-889X https://api.journalsdb.org/journals/0323-889X Fixed 1434-2944 Wiley forwards to Internation Review of Hydrobiology 1687-9996 https://api.journalsdb.org/journals/1687-9996 Manual add based on world cat https://www.worldcat.org/title/journal-of-the-egyptian-national-cancer-institute/oclc/1076662950&referer=brief_results
0955-3681 https://api.journalsdb.org/journals/0955-3681 Good 1932-0166 https://api.journalsdb.org/journals/1932-0166 No evidence ISSN exists 2164-5285 https://api.journalsdb.org/journals/2164-5285 Manual add based on world cat https://www.worldcat.org/title/national-vocational-guidance-bulletin/oclc/887954671&referer=brief_results
0003-8946 https://api.journalsdb.org/journals/0003-8946 Good doi, open access 0037-8968 https://api.journalsdb.org/journals/0037-8968 Good doi, open access 0197-5323 https://api.journalsdb.org/journals/0197-5323 Good old journal
0253-5904 https://api.journalsdb.org/journals/0253-5904 Fixed 0025-116X
0008-5286 https://api.journalsdb.org/journals/0008-5286 Good doi, open access 0031-5621 https://api.journalsdb.org/journals/0031-5621 Good old journal
0366-0796 https://api.journalsdb.org/journals/0366-0796 Good old journal
2043-7323 https://api.journalsdb.org/journals/2043-7323 Manual add based on world cat https://www.worldcat.org/title/advances-in-ophthalmology-and-vision-science/oclc/962546622&referer=brief_results
1752-9859 https://api.journalsdb.org/journals/1752-9859 Good 2475-0255 https://api.journalsdb.org/journals/2475-0255 Fixed 2475-0263
2590-1443 https://api.journalsdb.org/journals/2590-1443 Fixed 2210-7401
0967-0653 https://api.journalsdb.org/journals/0967-0653 Good, set publisher as Elsevier https://www.journals.elsevier.com/oceanographic-literature-review 1996-9481 https://api.journalsdb.org/journals/1996-9481 Fix, set publisher to Copernicus Publications 1996-9457 1878-013X https://api.journalsdb.org/journals/1878-013X Deleted, already part of 1755-599X 2472-4718 https://api.journalsdb.org/journals/2472-4718 Good doi, open access 0031-5710 https://api.journalsdb.org/journals/0031-5710 Good, old journal was later renamed 2234-8867 https://api.journalsdb.org/journals/2234-8867 Good doi, open access 1040-5712 https://api.journalsdb.org/journals/1040-5712 Fixed 1556-6617
0556-1892 https://api.journalsdb.org/journals/0556-1892 Good doi, open access 0255-2922 https://api.journalsdb.org/journals/0255-2922 Good, set publisher as Elsevier Related to 0254-6272 2043-7331 https://api.journalsdb.org/journals/2043-7331 Merge into wordcat title 2043-7323 https://www.worldcat.org/title/advances-in-ophthalmology-and-vision-science/oclc/828138938&referer=brief_results
0894-8631 https://api.journalsdb.org/journals/0894-8631 Good doi, open access 2193-0872 https://api.journalsdb.org/journals/2193-0872 Good doi 1344-7661 https://api.journalsdb.org/journals/1344-7661 Good, other journal with same title 1879-2871 https://api.journalsdb.org/journals/1879-2871 Good, likely continuation of other titles with same name 0738-0232 https://api.journalsdb.org/journals/0738-0232 Good 1608-4799 https://api.journalsdb.org/journals/1608-4799 Good doi, open access 0096-8013 https://api.journalsdb.org/journals/0096-8013 Good, old journal replaced by Journal of Geophysical Research 2194-6019 https://api.journalsdb.org/journals/2194-6019 Bad ISSN (never published) 0095-9502 https://api.journalsdb.org/journals/0095-9502 Fixed 0095-9774
0095-9510 https://api.journalsdb.org/journals/0095-9510 Good doi, open access 1521-3979 https://api.journalsdb.org/journals/1521-3979 Good 2331-3439 https://api.journalsdb.org/journals/2331-3439 Fixed 0095-9774
1522-9580 https://api.journalsdb.org/journals/1522-9580 Good 2210-9110 https://api.journalsdb.org/journals/2210-9110 Fixed 2210-9099
0254-6272 https://api.journalsdb.org/journals/0254-6272 Good open access
1096-9136 https://api.journalsdb.org/journals/1096-9136 Fixed 0742-3071
2234-5957 https://api.journalsdb.org/journals/2234-5957 Fixed 1226-086X
2029-0187 https://api.journalsdb.org/journals/2029-0187 Good doi, open access 2331-3420 https://api.journalsdb.org/journals/2331-3420 Good 1862-2720 https://api.journalsdb.org/journals/1862-2720 Good doi, open access
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ourresearch/journalsdb/issues/16#issuecomment-845381241, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABWPAJ7OMIZ6HHZXCXGUWTTOVKM7ANCNFSM447HS6WA .
I've attached the instructions on how to find missing ISSNs in case we need it. For now I will continue to work the incoming journals. But if it seems like it's a lot then maybe we can get some help.
That's great! Let us know if/when it seems like time to get help. Till then this flow seems to be working well!
all of these were in some recent customer upload files, and aren't in journalsdb or don't have data on dois_by_issued_year. Some of them might be for old journals. Can you add/update? Thanks!
https://api.journalsdb.org/journals/0005-2744 https://api.journalsdb.org/journals/6183-1638 https://api.journalsdb.org/journals/0955-3681 https://api.journalsdb.org/journals/1934-8800 https://api.journalsdb.org/journals/1755-599X https://api.journalsdb.org/journals/2213-3917 https://api.journalsdb.org/journals/1608-4799 https://api.journalsdb.org/journals/2234-5957 https://api.journalsdb.org/journals/1687-9996 https://api.journalsdb.org/journals/0254-6272 https://api.journalsdb.org/journals/1879-2871 https://api.journalsdb.org/journals/2210-9110 https://api.journalsdb.org/journals/1547-4127