search

bat-literature / bat-literature.github.io

The Bat Literature Project aims to facilitate discovery of scientific literature on bats (Chiroptera)
Creative Commons Zero v1.0 Universal
0 stars 0 forks source link

suspicious DOIs exist that are not redirected by https://doi.org #5

Closed jhpoelen closed 2 months ago

jhpoelen commented 4 months ago

In trying out all DOIs available in BatLit v0.1 , many redirected, however some did not. In total over 4.2k, and only about 28 DOIs causes a not found (404).

count status description
4202 302 temporary redirect
28 404 not found
2 301 permanent redirect
1 000 unknown

@ajacsherman @myrmoteras Please advise on how you'd like to prevent suspicious DOIs to propagate into Zenodo, possibly creating misdirected links .

404 https://doi.org/10.1007/s10393-017-1245-x; 404 https://doi.org/10.11694/pamj.supp.2015.22.1.6617 404 https://doi.org/10.32032/eid2206.160021 404 https://doi.org/10.1684/vir.2011.0418 404 https://doi.org/10.1371/currents.outbreaks.07992a87522e1f229c7cb023270a2af1 404 https://doi.org/10.1159/isbn.978-3-318-04030-2 404 https://doi.org/10.1644/09-MAMM-A-325.1.Key 404 https://doi.org/10.1002/path 404 https://doi.org/10.1182/blood-2014-02-514745.So 404 https://doi.org/10.1002/A 404 https://doi.org/10.4172/jgb.1000102 404 https://doi.org/10.1128/mBio.00164-10.Editor 404 https://doi.org/10.1371/journal. 000 pone.0200303 404 https://doi.org/10.1261/rna.046011.114.ing 404 https://doi.org/10.3201/eid1706.101355 404 https://doi.org/10.1007/s13398-014-0173-7.2 301 https://doi.org/10.3201/eid0703.010312 404 https://doi.org/10.1038/ncomms2343.Functional 404 https://doi.org/10.1016/j.immuni.2011.06.005.Skin-Resident 404 https://doi.org/10.1002/jcla 301 https://doi.org/10.1172/JCI200423215 404 https://doi.org/10.1128/AEM.70.5.2823 404 https://doi.org/10.1172/32108.of 404 https://doi.org/10.1124/pr.58.3.10. 404 https://doi.org/10.1101/gr.191049.115.Freely 404 https://doi.org/10.1002/pbc 404 https://doi.org/10.2307/2346101 404 https://doi.org/10.1128/CDLI.11.2.344 404 https://doi.org/10.1146/annurev.ecolsys.36.102003.152 404 https://doi.org/10.1124/jpet.105.095976.stic

ajacsherman commented 3 months ago

Some of these links (like https://doi.org/10.1007/s10393-017-1245-x) bring you to the data source. For the ones that are broken links, I would recommend omitting them. Donat?

jhpoelen commented 3 months ago

Thanks for having a look at the suspicious DOIs.

Note that https://doi.org/10.1007/s10393-017-1245-x; (note the trailing semicolon) appears in the Bat Lit records and does not resolve. However, as you noted, when the semi-colon is truncated, the doi does appear to resolve. This still leaves the original DOI generating a "Error: DOI not found" message.

Please note if there's any other DOis in the list that resolve for you, but didn't resolve for me. I checked their status automatically, but perhaps I made some kind of mistake in my DOI checking method.

See attached screenshot.

image

ajacsherman commented 3 months ago

Can you find where https://doi.org/10.1684/vir.2011.0418 came from? The name of the paper, please?

ajacsherman commented 3 months ago

Would you provide where https://doi.org/10.1159/isbn.978-3-318-04030-2 came from? The citation please?

ajacsherman commented 3 months ago

Searching by the DOI is not producing results. Would you share whatever citations you have associated with these corrupt links?

ajacsherman commented 3 months ago

I was successful with https://doi.org/10.3201/eid0703.010312 [https://doi.org/10.3201%2Feid0703.010312] and https://doi.org/10.11694/pamj.supp.2015.22.1.6617 [https://doi.org/10.11604/pamj.supp.2015.22.1.6617] if you need a examples.

myrmoteras commented 3 months ago

@ajacsherman you can use https://refindit.org/ to search for DOI by entering eg the title etc.

myrmoteras commented 3 months ago

Would you provide where https://doi.org/10.1159/isbn.978-3-318-04030-2 has this https://karger.com/books/book/920/Virus-Infections-in-Bats found by searching via google

ajacsherman commented 3 months ago

I don't have the titles associated with the corrupt links

ajacsherman commented 3 months ago

"Would you provide where https://doi.org/10.1159/isbn.978-3-318-04030-2 has this https://karger.com/books/book/920/Virus-Infections-in-Bats found by searching via google" I'm not sure what you mean? Was this for Jorrit?

jhpoelen commented 3 months ago

Can you find where https://doi.org/10.1684/vir.2011.0418 came from? The name of the paper, please?

This DOI was part of batlit v0.1, but is no longer present in batlit v0.2 .

v0.1 - 

preston ls --anchor hash://sha256/6ba3d79cf1fd6349012cb4e527b6727b3e41e140489fa9c02f132e2cdd88d189\
 | grep hasVersion\
 | grep items?\
 | preston cat\
 | jq -c .[]\
 | grep "10.1684/vir.2011.0418"\
 | jq .

yielded:

{
  "key": "UP3ETFVG",
  "version": 691,
  "library": {
    "type": "group",
    "id": 5435545,
    "name": "Bat Literature Project",
    "links": {
      "alternate": {
        "href": "https://www.zotero.org/groups/bat_literature_project",
        "type": "text/html"
      }
    }
  },
  "links": {
    "self": {
      "href": "https://api.zotero.org/groups/5435545/items/UP3ETFVG",
      "type": "application/json"
    },
    "alternate": {
      "href": "https://www.zotero.org/groups/bat_literature_project/items/UP3ETFVG",
      "type": "text/html"
    }
  },
  "meta": {
    "createdByUser": {
      "id": 6296343,
      "username": "deeannreeder",
      "name": "",
      "links": {
        "alternate": {
          "href": "https://www.zotero.org/deeannreeder",
          "type": "text/html"
        }
      }
    },
    "creatorSummary": "Talbi and Bourhy",
    "parsedDate": "2011-09-01",
    "numChildren": 1
  },
  "data": {
    "key": "UP3ETFVG",
    "version": 691,
    "itemType": "journalArticle",
    "title": "Dog rabies in Africa through genetic, spatial and temporal analysis of the isolates",
    "creators": [
      {
        "creatorType": "author",
        "firstName": "Chiraz",
        "lastName": "Talbi"
      },
      {
        "creatorType": "author",
        "firstName": "Herve",
        "lastName": "Bourhy"
      }
    ],
    "abstractNote": "This work illustrates the studies that have provided a better understanding of the genetic and geographic structure of dog rabies virus and the dynamics of rabies in domestic dogs in Africa, the second largest continent most affected by this disease. These investigations are a key element to control the disease and identify effective strategies for eliminating rabies locally, nationally as well as internationally and could drastically reduce human deaths caused by this disease. They also allow for the first time the investigation of the role of humans in the canine rabies virus spreading.",
    "publicationTitle": "Virologie",
    "volume": "15",
    "issue": "",
    "pages": "307-318",
    "date": "September 1, 2011",
    "series": "",
    "seriesTitle": "",
    "seriesText": "",
    "journalAbbreviation": "Virologie",
    "language": "",
    "DOI": "10.1684/vir.2011.0418",
    "ISSN": "",
    "shortTitle": "",
    "url": "",
    "accessDate": "",
    "archive": "",
    "archiveLocation": "",
    "libraryCatalog": "ResearchGate",
    "callNumber": "",
    "rights": "",
    "extra": "",
    "tags": [],
    "collections": [
      "DZKBQXJR"
    ],
    "relations": {
      "owl:sameAs": "http://zotero.org/groups/2719577/items/5VD54MKG"
    },
    "dateAdded": "2024-03-07T00:46:26Z",
    "dateModified": "2024-03-07T00:46:26Z"
  }
}

Note, however, than an attachment related to the v0.1 reference does exist in v0.2 - 

preston ls --algo md5 --anchor hash://md5/be692b93a8edde4c4269be9e7d4ec1d7 | grep hasVersion | grep items? | preston cat | jq -c .[] | grep "UP3ETFVG" | jq .

yielded:

{
  "key": "RENUCU5P",
  "version": 691,
  "library": {
    "type": "group",
    "id": 5435545,
    "name": "Bat Literature Project",
    "links": {
      "alternate": {
        "href": "https://www.zotero.org/groups/bat_literature_project",
        "type": "text/html"
      }
    }
  },
  "links": {
    "self": {
      "href": "https://api.zotero.org/groups/5435545/items/RENUCU5P",
      "type": "application/json"
    },
    "alternate": {
      "href": "https://www.zotero.org/groups/bat_literature_project/items/RENUCU5P",
      "type": "text/html"
    },
    "up": {
      "href": "https://api.zotero.org/groups/5435545/items/UP3ETFVG",
      "type": "application/json"
    }
  },
  "meta": {
    "createdByUser": {
      "id": 6296343,
      "username": "deeannreeder",
      "name": "",
      "links": {
        "alternate": {
          "href": "https://www.zotero.org/deeannreeder",
          "type": "text/html"
        }
      }
    }
  },
  "data": {
    "key": "RENUCU5P",
    "version": 691,
    "parentItem": "UP3ETFVG",
    "itemType": "attachment",
    "linkMode": "linked_url",
    "title": "ResearchGate Link",
    "accessDate": "2021-01-22T18:00:13Z",
    "url": "https://www.researchgate.net/publication/286280182_Dog_rabies_in_Africa_through_genetic_spatial_and_temporal_analysis_of_the_isolates",
    "note": "",
    "contentType": "",
    "charset": "",
    "tags": [],
    "relations": {
      "owl:sameAs": "http://zotero.org/groups/2719577/items/W2XXQ47K"
    },
    "dateAdded": "2024-03-07T00:46:26Z",
    "dateModified": "2024-03-07T00:46:26Z"
  }
}

@ajacsherman any idea why reference described in v0.1 https://www.zotero.org/groups/bat_literature_project/items/UP3ETFVG no longer exists?

jhpoelen commented 3 months ago

Would you provide where https://doi.org/10.1159/isbn.978-3-318-04030-2 came from? The citation please?

also exists in v0.1, but not in v0.2 

preston ls --anchor hash://sha256/6ba3d79cf1fd6349012cb4e527b6727b3e41e140489fa9c02f132e2cdd88d189 | grep hasVersion | grep items? | preston cat | jq -c .[] | grep "10.1159/isbn.978-3-318-04030-2" | jq .

yielded 

{
  "key": "GELIBCEP",
  "version": 608,
  "library": {
    "type": "group",
    "id": 5435545,
    "name": "Bat Literature Project",
    "links": {
      "alternate": {
        "href": "https://www.zotero.org/groups/bat_literature_project",
        "type": "text/html"
      }
    }
  },
  "links": {
    "self": {
      "href": "https://api.zotero.org/groups/5435545/items/GELIBCEP",
      "type": "application/json"
    },
    "alternate": {
      "href": "https://www.zotero.org/groups/bat_literature_project/items/GELIBCEP",
      "type": "text/html"
    }
  },
  "meta": {
    "createdByUser": {
      "id": 6296343,
      "username": "deeannreeder",
      "name": "",
      "links": {
        "alternate": {
          "href": "https://www.zotero.org/deeannreeder",
          "type": "text/html"
        }
      }
    },
    "creatorSummary": "Sulkin and Allen",
    "parsedDate": "1974",
    "numChildren": 0
  },
  "data": {
    "key": "GELIBCEP",
    "version": 608,
    "itemType": "journalArticle",
    "title": "Virus infections in bats",
    "creators": [
      {
        "creatorType": "author",
        "firstName": "S E",
        "lastName": "Sulkin"
      },
      {
        "creatorType": "author",
        "firstName": "R",
        "lastName": "Allen"
      }
    ],
    "abstractNote": "",
    "publicationTitle": "Monographs in virology",
    "volume": "8",
    "issue": "0",
    "pages": "1-103",
    "date": "1974",
    "series": "",
    "seriesTitle": "",
    "seriesText": "",
    "journalAbbreviation": "",
    "language": "",
    "DOI": "10.1159/isbn.978-3-318-04030-2",
    "ISSN": "0077-0965",
    "shortTitle": "",
    "url": "",
    "accessDate": "",
    "archive": "",
    "archiveLocation": "",
    "libraryCatalog": "",
    "callNumber": "",
    "rights": "",
    "extra": "PMID: 4367453\nISBN: 0077-0965 (Print) 0077-0965 (Linking)",
    "tags": [],
    "collections": [
      "DZKBQXJR"
    ],
    "relations": {
      "owl:sameAs": "http://zotero.org/groups/2446996/items/L845NLV5"
    },
    "dateAdded": "2024-03-06T17:28:04Z",
    "dateModified": "2024-03-06T17:28:04Z"
  }
}

however, no such record was found in v0.2.

@ajacsherman curious to hear why this record no longer exists. . .

jhpoelen commented 3 months ago

I was successful with https://doi.org/10.3201/eid0703.010312 [https://doi.org/10.3201%2Feid0703.010312] and https://doi.org/10.11694/pamj.supp.2015.22.1.6617 [https://doi.org/10.11604/pamj.supp.2015.22.1.6617] if you need a examples.

Hey @ajacsherman - thanks for sharing your examples.

From what I can tell, the suspicious DOIs that you could no longer find have been deleted from the Zotero group somehow.

This is consistent with finding 10.3201/eid0703.010312 in v0.2 as well as in v0.1.

preston ls --algo md5 --anchor hash://md5/be692b93a8edde4c4269be9e7d4ec1d7 | grep hasVersion | grep items? | preston cat | jq -c .[] | grep "10.3201/eid0703.010312" | jq . 
{
  "key": "UNPAII9M",
  "version": 405,
  "library": {
    "type": "group",
    "id": 5435545,
    "name": "Bat Literature Project",
    "links": {
      "alternate": {
        "href": "https://www.zotero.org/groups/bat_literature_project",
        "type": "text/html"
      }
    }
  },
  "links": {
    "self": {
      "href": "https://api.zotero.org/groups/5435545/items/UNPAII9M",
      "type": "application/json"
    },
    "alternate": {
      "href": "https://www.zotero.org/groups/bat_literature_project/items/UNPAII9M",
      "type": "text/html"
    },
    "attachment": {
      "href": "https://api.zotero.org/groups/5435545/items/6GC4864A",
      "type": "application/json",
      "attachmentType": "application/pdf",
      "attachmentSize": 76993
    }
  },
  "meta": {
    "createdByUser": {
      "id": 6296343,
      "username": "deeannreeder",
      "name": "",
      "links": {
        "alternate": {
          "href": "https://www.zotero.org/deeannreeder",
          "type": "text/html"
        }
      }
    },
    "creatorSummary": "Yob et al.",
    "parsedDate": "2001-06",
    "numChildren": 2
  },
  "data": {
    "key": "UNPAII9M",
    "version": 405,
    "itemType": "journalArticle",
    "title": "Nipah virus infection in bats (order Chiroptera) in peninsular Malaysia",
    "creators": [
      {
        "creatorType": "author",
        "firstName": "J. M.",
        "lastName": "Yob"
      },
      {
        "creatorType": "author",
        "firstName": "H.",
        "lastName": "Field"
      },
      {
        "creatorType": "author",
        "firstName": "A. M.",
        "lastName": "Rashdi"
      },
      {
        "creatorType": "author",
        "firstName": "C.",
        "lastName": "Morrissy"
      },
      {
        "creatorType": "author",
        "firstName": "B.",
        "lastName": "van der Heide"
      },
      {
        "creatorType": "author",
        "firstName": "P.",
        "lastName": "Rota"
      },
      {
        "creatorType": "author",
        "firstName": "A.",
        "lastName": "bin Adzhar"
      },
      {
        "creatorType": "author",
        "firstName": "J.",
        "lastName": "White"
      },
      {
        "creatorType": "author",
        "firstName": "P.",
        "lastName": "Daniels"
      },
      {
        "creatorType": "author",
        "firstName": "A.",
        "lastName": "Jamaluddin"
      },
      {
        "creatorType": "author",
        "firstName": "T.",
        "lastName": "Ksiazek"
      }
    ],
    "abstractNote": "Nipah virus, family Paramyxoviridae, caused disease in pigs and humans in peninsular Malaysia in 1998-99. Because Nipah virus appears closely related to Hendra virus, wildlife surveillance focused primarily on pteropid bats (suborder Megachiroptera), a natural host of Hendra virus in Australia. We collected 324 bats from 14 species on peninsular Malaysia. Neutralizing antibodies to Nipah virus were demonstrated in five species, suggesting widespread infection in bat populations in peninsular Malaysia.",
    "publicationTitle": "Emerging Infectious Diseases",
    "volume": "7",
    "issue": "3",
    "pages": "439-441",
    "date": "2001 May-Jun",
    "series": "",
    "seriesTitle": "",
    "seriesText": "",
    "journalAbbreviation": "Emerging Infect. Dis.",
    "language": "eng",
    "DOI": "10.3201/eid0703.010312",
    "ISSN": "1080-6040",
    "shortTitle": "",
    "url": "",
    "accessDate": "",
    "archive": "",
    "archiveLocation": "",
    "libraryCatalog": "PubMed",
    "callNumber": "",
    "rights": "",
    "extra": "PMID: 11384522\nPMCID: PMC2631791",
    "tags": [
      {
        "tag": "Animals",
        "type": 1
      },
      {
        "tag": "Antibodies, Viral",
        "type": 1
      },
      {
        "tag": "Chiroptera",
        "type": 1
      },
      {
        "tag": "Malaysia",
        "type": 1
      },
      {
        "tag": "Paramyxoviridae Infections",
        "type": 1
      },
      {
        "tag": "Paramyxovirinae",
        "type": 1
      },
      {
        "tag": "Seroepidemiologic Studies",
        "type": 1
      }
    ],
    "collections": [
      "DZKBQXJR"
    ],
    "relations": {
      "owl:sameAs": "http://zotero.org/groups/2446996/items/PAA8W8JH"
    },
    "dateAdded": "2024-03-06T17:26:09Z",
    "dateModified": "2024-03-06T17:26:09Z"
  }
}
jhpoelen commented 3 months ago

@ajacsherman so, as far as I can tell, if you can't find suspicious DOIs via Zotero, they have been either removed or updated.

Please note that doi 10.3201/eid0703.010312 made it to the list, because they had a 301 redirect instead of an expected 302 redirect. In other words, the doi url https://doi.org/10.3201/eid0703.010312 did redirect, only with an unusual redirect code 301.

Please holler if you'd like me to clarify by voice chat, happy to do so.

If you managed to get through all suspicious DOIs for this issue #5 , please feel free to close this issue.

Thanks for all your careful curation work.

ajacsherman commented 3 months ago

Good morning, I was replacing the DOIs I could find that were problematic. If I can't find the links you provided, I assume I have fixed them. Would you be able to run your report again with the corrupt DOIs as well as the citation associated with them? I should have kept a list of the ones I edited.

jhpoelen commented 3 months ago

@ajacsherman Thanks for your efforts.

Does that also include the DOIs in #4 ? For some reason, the #4 reference to a DOI value associated with https://www.zotero.org/groups/5435545/bat_literature_project/items/2WK8GVE9 didn't change yet.

I'd be happy to re-run the check for the v0.3 release of batlit. Please let me know when would be a good time to run.

Alternatively, I can re-run the DOI check on batlit v0.2 and include the links to their associated Zotero records.

Please let me know what works best for you.

jhpoelen commented 3 months ago

Here's a list of some of the 404 (not redirected DOIs) and their associated Zotero URLs.

via 

preston ls\
 --algo md5\
 --anchor hash://md5/be692b93a8edde4c4269be9e7d4ec1d7\
 --remote https://linker.bio\
 | grep hasVersion\
 | grep "items?"\
 | preston cat\
 | jq -c '.[]'\
 | grep -f 404.txt\
  | jq --raw-output '[ .data.DOI, .links.alternate.href ] | @tsv'\
 | mlr --itsvlite --omd cat

with 404.txt being

10.1007/s10393-017-1245-x;
10.11694/pamj.supp.2015.22.1.6617
10.32032/eid2206.160021
10.1684/vir.2011.0418
10.1371/currents.outbreaks.07992a87522e1f229c7cb023270a2af1
10.1159/isbn.978-3-318-04030-2
10.1644/09-MAMM-A-325.1.Key
10.1002/path
10.1182/blood-2014-02-514745.So
10.1002/A
10.4172/jgb.1000102
10.1128/mBio.00164-10.Editor
pone.0200303
10.1261/rna.046011.114.ing
10.3201/eid1706.101355
10.1007/s13398-014-0173-7.2
10.1038/ncomms2343.Functional
10.1016/j.immuni.2011.06.005.Skin-Resident
10.1002/jcla
10.1128/AEM.70.5.2823
10.1172/32108.of
10.1124/pr.58.3.10.
10.1101/gr.191049.115.Freely
10.1002/pbc
10.2307/2346101
10.1128/CDLI.11.2.344
10.1146/annurev.ecolsys.36.102003.152
10.1124/jpet.105.095976.stic
suspicious doi Zotero record URL
10.1371/journal. pone.0200303 https://www.zotero.org/groups/bat_literature_project/items/TE5RIWRU
10.1007/s10393-017-1245-x; https://www.zotero.org/groups/bat_literature_project/items/ZUHUHEHZ
10.11694/pamj.supp.2015.22.1.6617 https://www.zotero.org/groups/bat_literature_project/items/RDMWKGTG
10.32032/eid2206.160021 https://www.zotero.org/groups/bat_literature_project/items/4A7PYESU
10.1371/currents.outbreaks.07992a87522e1f229c7cb023270a2af1 https://www.zotero.org/groups/bat_literature_project/items/JT3HBLUQ
10.1644/09-MAMM-A-325.1.Key https://www.zotero.org/groups/bat_literature_project/items/LL5T2AXJ
10.1002/jcla https://www.zotero.org/groups/bat_literature_project/items/MMJLU8M9
ajacsherman commented 3 months ago

This helps! Do you have the complete list with links? Resolving now...

jhpoelen commented 3 months ago

@ajacsherman happy to hear that the list of 404 DOIs and their associated Zotero links helped.

There's the other list with likely malformed DOIs also in related issue at #4 . Did you include these in addition to the DOIs that did not resolve?

jhpoelen commented 3 months ago

We can always catch others in the next review. . . I am sure that many things have changed. I see these reviews like maintaining a garden . . . always weeds to be pulled, plants to be cared for, and new plants to be planted.

ajacsherman commented 3 months ago

I love this analogy!

jhpoelen commented 2 months ago

closing this issue to make room for fresh doi review.