Support short DOI names

[bwiernik]

The search page doesn't appear to support short DOI names (http://shortdoi.org)-- it shows the "can't type, must paste" error when I try to past a short DOI.

e.g., 10/895

[jasonpriem]

Thanks! Alas supporting shortDOI is not this simple. We are using the DOI prefixes to help us determine which DOIs are DataCite ones rather than CrossRef ones, which is in turn necessary to know which APIs to call to resolve them. So, when we accept a shortDOI, we'll need to also do an extra lookup to get the canonical version with its prefix info.

This can, of course, be done. It's not that hard. However, it's extra work to implement, extra code surface area, and extra latency on each request. So we'd like to wait on this one for now. Once we've heard a few more people +1 shortDOI support we'll bump it up in the priority queue. Will leave this issue open to see if it gets more traction.

In the meantime we will supply a more useful error message at least, the current one sucks for this use case.

Thanks again for flagging this up, and for the PR.

[bwiernik]

One solution to support shortDOI would be to query the DOI API to get then long DOI, then pass this to the rest of the oaDOI functions.

So, if a "10/" string is detected inputVal.match(/10\/[^\s]*[^\s\.,]/), extract the shortDOI from the input, then pass it to a function like this:

var url = 'https://doi.org/api/handles/' + encodeURIComponent(shortDOI); 
var req = new XMLHttpRequest();
req.open('GET', url, true);
req.responseType = 'json';

req.onreadystatechange = function() { 
  if (req.readyState == 4) {
    if (req.status == 200) {
      if (req.response.responseCode == 1) {
        var longDOI = req.response.values["1"].data.value;
        return longDOI;
      } else if (req.status == 404) {
        //invalid DOI response
      } else {
        //no response
      }
    }
  }
};

req.send(null);

I'd be happy to submit a pull request for this, but I'm having a hard time navigating the repo and finding where this processing happens.

[jasonpriem]

@bwiernik Good idea! However, the oaDOI page in question is no longer up.

[bwiernik]

@jasonpriem Yes, I'm aware of that. But the oaDOI/unpaywall resolver still works to search for open access texts via OpenURL. Currently, if the resolver receives a shortDOI, it fails silently and redirects to the unpaywall.org home page. I think a better behavior would be to retrieve the long DOI and the resolve it. Could you point me to the correct repo to look at the oadoi resolver behavior?

[jasonpriem]

Ah, understood. Yes, good point...in fact pretty much everything Unpaywall does (Unpaywall is the new name for oaDOI, which we need to change on this repository, oops...) fails on shortDOIs. We have found they are just not used enough in the wild, particularly for our use cases, to justify support.

I like your solution for low-volume cases, but unfortunately I don't think it's practical for the scale we're supporting right now....we have a completely open API (no API keys, no IP-based throttling) and we get millions of requests daily. So we need to avoid putting anything that requires something as (relatively) slow as a call out to Crossref in front of an API return. We used to do that kind of hot-calling and it scaled very poorly. If someone wanted to throw a million shortDOIs at us in a few hours, now we have to make a million calls out to Crossref, and not only is that going to challenge our infrastructure, it's not great for them either.

So, in order to integrate shortDOIs, the only practical source I can think of would be to bring every one of them into our database and index them all beforehand. I think this is what we'll do if there's ever enough demand to support them.

[bwiernik]

Got it. That makes sense. This came up because some users of my Zotero plugin for retrieving short DOIs noticed that they could not longer call Unpaywall using Zotero's OpenURL function once they had replaced DOIs with shortDOIs. It would definitely be nice if you guys could support them, but obviously a relatively small group of users.

[bwiernik]

In any event, it might be nice if Unpaywall failed with a more informative message when given a shortDOI.

[jasonpriem]

@bwiernik good point on failing with a more useful error. I've opened #56 to address this.