rdmpage commented 5 years ago

The Unpaywall browser extension (Chrome and Firefox) doesn't work with BHL. If a user visits the page for an article an it is behind a paywall, e.g. https://doi.org/10.1080/00222932208632640, and there is a legally free to access version online elsewhere, then the extension will display a green "lock" symbol and if the user clicks on that they will be taken to that version. The article https://doi.org/10.1080/00222932208632640 (III.—Some new species of earthworms belonging to the genus Glyphidrilus) is in BHL, so the Unpaywall lock should be green, but it is not.

rdmpage commented 5 years ago

Investigating with @nicolekearney revealed that the BHL OAI-PMH endpoint wasn't registered with Unpaywall, so we registered it. The endpoint is acceptable to Unpaywall, as shown by passing their test: https://api.unpaywall.org/repository/endpoint/test/https://www.biodiversitylibrary.org/oai

{
    "results": {
        "check0_identify_status": "SUCCESS!",
        "check1_query_status": "SUCCESS!",
        "sample_pmh_record": "{\"contributor\": [\"MBLWHOI Library\"], \"language\": [\"German\"], \"description\": [null], \"subject\": [\"Chlorophyll\", \"Spectra\"], \"publisher\": [\"Stuttgart,Schweizerbart,1872.\"], \"identifier\": [\"https://www.biodiversitylibrary.org/item/16157\", \"info:doi/10.5962/bhl.title.1311\"], \"creator\": [\"Kraus, Gregor,      1841-1915\"], \"type\": [\"text\", \"Book\"], \"title\": [\"Zur Kenntniss der Chlorophyllfarbstoffe und ihrer Verwandten; spectralanalytische Untersuchungen. \"], \"rights\": [\"Public domain.  The BHL considers that this work is no longer under copyright protection.\"]}"
    }
}

So new we wait while Unpaywall indexes BHL. We can check on the status of a test record using Unpaywall's API on some test DOIs.

10.1111/j.1096-3642.1818.tb00336.x
10.1636/A08-27.1
10.1080/00222932208632640
10.3417/2010131 (already available from Harvard DASH).

rdmpage commented 5 years ago

Can monitor progress here: https://unpaywall.org/sources/repository/q6caunfjwsunh6wiqwpg

rdmpage commented 5 years ago

Table showing progress of harvesting.

Key	06-16
Number of OAI-PMH records with a unique title	644361
Number that match a published article DOI and have full text freely available, by version	53633
publishedVersion	4602
acceptedVersion	0
submittedVersion	49031

crowleyb commented 5 years ago

BHL originally registered with Unpaywall on April 3, 2019. As of April 15 our status was as follows: But now it is different: https://unpaywall.org/sources/repository/fsxfk6gcvszjgobj4jnt

1 We'll need to see what happened with the first batch that was registered and why it changed.

2 it will also be useful now to make sure there is only one instance of BHL represented in Unpaywall

nicolekearney commented 5 years ago

Dear Heather, Jason and Richard,

We have been eagerly awaiting the matching of the DOIs in BHL via Unpaywall. I keep checking the Thylacine example: https://doi.org/10.1111/j.1096-3642.1818.tb00336.x (behind a paywall on Wiley, open access on BHLhttps://www.biodiversitylibrary.org/part/5582#/summary: Unpaywall is still not picking up the open access version in BHL from the Wiley website).

In the meantime we have received the message below from Bianca Crowley at BHL, who informed us that BHL was originally registered with Unpaywall in April 2019 and that by the 15 April, Unpaywall had matched 56,000+ DOIs with a freely accessible version in BHL (we were unaware of this when we reregistered BHL this month). The new endpoint we have been trackinghttps://unpaywall.org/sources/repository/q6caunfjwsunh6wiqwpg now has 95,000+ matches.

We are concerned that, if the BHL DOIs have been matched since April, why is the open access content still not being picked up by Unpaywall? We’re also wondering if it is a problem that there are now two instances of BHL in Unpaywall with two different endpoints: http://www.biodiversitylibrary.org/oai

https://www.biodiversitylibrary.org/oai

We’re extremely keen to see the BHL content discoverable via Unpaywall as soon as possible and we greatly appreciate your time trying to resolve this. Please let us know if there is anything we can do to help.

Kind regards, Nicole (and Rod)

crowleyb commented 5 years ago

Hi Nicole, Rod,

In case this information is useful, I believe there are over 72,000 DOIs for segments in BHL. Please see https://admin.biodiversitylibrary.org/ReportDOIByInstitution.aspx for more information. Mike would be able to confirm the actual numbers.

Please forgive my confusion about Unpaywall and DOIs in general, but I do not understand the example below regarding https://www.biodiversitylibrary.org/part/5582#/summary. This article in BHL has a DOI but this is the DOI that Wiley assigned to their copy of the article behind their paywall (which is totally unjust b/c it’s a PD work but I digress…). What is the expected functionality via Unpaywall?

Thank you for reaching out to Unpaywall for us.

Kind regards, Bianca

nicolekearney commented 5 years ago

Hi Bianca,

Have you downloaded the Unpaywall extension? You can do so by clicking on the “Get the Extension” button on the Unpaywall homepagehttps://unpaywall.org/ (note it only works in Chrome).

Once you have the extension, go to the definitive (DOI’d) version of the article on Wiley: https://doi.org/10.1111/j.1096-3642.1818.tb00336.x

You should see a grey locked lock symbol on the right hand side of the page. If you click on the lock, you will get the message “The Unpaywall extension couldn't find any legal open-access version of this article.”

This is what isn’t working. Because we know there is a legal open-access version of this article – on BHL. And because that article has the DOI displayed on the landing page so it should be discoverable via Unpaywall.

Once/if Unpaywall can find it, the lock symbol on the Wiley website will be unlocked and green, and clicking on it will take you directly to the open access version on BHL.

We want the Unpaywall extension to work for all the tens of thousands of open access versions of articles on BHL so that when you’re on a paywalled version you can link directly to the content on the BHL website.

I hope that answers your question. Nicole

crowleyb commented 5 years ago

Thanks Nicole. I have downloaded the Unpaywall extension and I think things make more sense now. I look forward to hearing what they say about a fix.

nicolekearney commented 5 years ago

Dear Richard, Jason and Heather,

Is it possible for you to answer the following questions (from my email below) so we at least know where we’re at:

Is there a reason that the Unpaywall extension still can’t find an open-access version of this article https://doi.org/10.1111/j.1096-3642.1818.tb00336.x (behind a paywall on Wiley), even though there is an open access version on BHLhttps://www.biodiversitylibrary.org/part/5582#/summary (this is just one example of so many).
Knowing that the BHL content was originally registered with Unpaywall in April, it’s surely not just a matter of just checking every day in the hope that one day it might start working. Or does it really take this long?
Are you able to look into why it isn’t working?

Thank you again for your time, Nicole

Nicole Kearney Manager, Biodiversity Heritage Library Australia

nicolekearney commented 5 years ago

An update: we have been trying to get an answer from Unpaywall as to why their plug-in still isn't able to locate the content in BHL, despite our testing examples where the DOIs are included on the landing pages for articles. We keep being told that it's just a matter of waiting and they've asked us to check back every day to see if it's working yet. When we discovered that BHL was registered with Unpaywall in April, we were more baffled because that seems like a very long time to wait.

It would be amazing if we could get this work.

Our primary reasons for pushing this are:

the commercial websites (e.g. Wiley, Taylor & Francis, Oxford Academic, etc) have assigned DOIs to a significant proportion of the out-of-copyright literature that BHL has made freely accessible online;
everyone who publishes using DOIs (pretty much all modern scientific publishers) has to include these DOIs in the citations in their publications;
these DOIs are links and thus everyone clicking on those links is directed to the versions behind paywalls;
the Unpaywall plugin is extremely useful in these cases - it searches for freely accessible versions available elsewhere. If it finds one, a green unlock symbol appears on the right hand side of your screen. Clicking on that unlock symbol takes you directly to the open access version.
BUT the Unpaywall plugin is still not working for BHL content: when you're on a paywalled copy of something that is freely accessible on BHL, Unpaywall reports that there are NO open access versions online (unless someone other than BHL has an open access copy).
In most cases the copies on BHL are much better quality than those on the commercial websites (which are usually black and white and horribly grainy). It's very sad for the user that BHL's versions are not the ones with the DOIs and that they are undiscoverable via Unpaywall.
So, for example, if I were to write an article about the Thylacine and cite its first description (1808), I would have to include the DOI in my reference list. The DOI'd version of this article is behind a paywall. Anyone clicking on the DOI in my reference list would be taken to the paywalled version: https://doi.org/10.1111/j.1096-3642.1818.tb00336.x. If the Unpaywall plugin worked for BHL content, a green unlock symbol would appear on the paywalled version. Clicking on the green unlock symbol would take you directly to the beautifully-scanned open version on BHL: https://www.biodiversitylibrary.org/part/5582#/summary
DOIs are now considered an essential part of bibliographic metadata and are rapidly being added to citations everywhere. This includes Wikipedia. This means that all Wikipedia users are being directed to the Paywalled versions of these out-of-copyright articles. For example, if you search for "Thylacine" in Google, the first result returned is for the Wikipedia page. The citation includes the DOI (as it is supposed to), which links directly to the paywalled version. Unpaywall would allow people to link directly to the version on BHL (from the locked version).

Note: you need to download the Unpaywall plugin for this to work: https://unpaywall.org/ (note it only works in Chrome and Firefox).

crowleyb commented 5 years ago

Nicole, Rod, thank you for continuing to push this with Unpaywall. Would it be helpful if I chimed in to the email with Richard, Jason, and Heather to extra reiterate that we would like to get this all worked out? I’m not sure how squeaky you want this wheel to be… =)

Bianca

nicolekearney commented 5 years ago

@crowleyb @rdmpage Yes please squeak as hard as you can squeak. It would be such a wonderful thing if we could make the all the wonderful content on BHL discoverable by Unpaywall.

nicolekearney commented 5 years ago

Hi Nicole, Thank you for your patience and I'm sorry for not replying sooner. I've been waiting until the problem is solved, which it isn't yet, but I do want to at least let you know what the problem is and that we haven't forgotten about it. Out problem with the example you provided, and others like https://www.biodiversitylibrary.org/part/281352, is that on repository pages we look for a PDF link to confirm that the document is actually available. We're not able to recognize the embedded reader on those pages, so they don't get used. Does the "View article" link always lead to a full copy? If so, I think we can just add a special case for these pages. The duplicate endpoint doesn't create a problem aside from some wasted harvesting effort on our end, but I'll go ahead and remove one. https://unpaywall.org/sources/repository/q6caunfjwsunh6wiqwpg will be the one we keep. Thanks, Richard Orr Lead Developer, Unpaywall Impactstory: We make tools to power the Open Science revolution https://support.unpaywall.org/public/tickets/130addea2170edd9e4fc198708090fbfc759e801fa0bc184f45c8828c721b636

          On
          Sun, 30 Jun at  6:00 PM
          ,  Nicole Kearney <nkearney@museum.vic.gov.au>  wrote:
           Dear Richard, Jason and Heather,  Is it possible for you to answer the following questions (from my email below) so we at least know where we’re at:     1.  Is there a reason that the Unpaywall extension still can’t find an open-access version of this article https://doi.org/10.1111/j.1096-3642.1818.tb00336.x (behind a paywall on Wiley), even though there is an open access version on BHL<https://www.biodiversitylibrary.org/part/5582#/summary> (this is just one example of so many).      1.  Knowing that the BHL content was originally registered with Unpaywall in April, it’s surely not just a matter of just checking every day in the hope that one day it might start working. Or does it really take this long?      1.  Are you able to look into why it isn’t working?   Thank you again for your time, Nicole

nicolekearney commented 5 years ago

Dear Richard,

Thank you for getting back to us and explaining the issue. I can confirm that all of BHL content is available for free and open access and as such the “View Article” link will always lead to the full text of the content. Please add a special case for all BHL pages as you suggest below.

We would very much appreciate hearing from you once this has been completed.

Thank you again for taking the time to review our content, eliminate the duplicate entry, and granting us an exception to the rule. Please let me know if you run into any further issues with BHL content and I will do my best to assist.

Regards, Bianca

Bianca Crowley Digital Collections Manager Digital Programs & Initiatives Division crowleyb@si.edumailto:crowleyb@si.edu | 202.633.2239

[EmailSignature_option1_noTag_RGB]

rdmpage commented 5 years ago

Ah, the comment by @richard-orr makes sense, I'm assuming that Unpaywall looks for things such as the citation_pdf_url tag to locate the PDF. BHL pages don't include this tag (or anything else machine readable that links to a PDF). Given that (most?) readers just want to read the PDF it would ultimately be useful if BHL had pre-generated articles reading to deliver to the reader. I started some work on this by generating PDFs for BioStor articles and storing them on Internet Archive. Maybe @crowleyb can comment on whether BHL can generate all article PDFs so that, from a user's perspective, BHL gives them what they most likely want.

@nicolekearney as an aside I think this is another reason to add Google Scholar tags to the Memoirs pages, because at the moment those pages lack the Unpaywall lock symbol.

crowleyb commented 5 years ago

It’s a recent addition but yes BHL can general article PDFs. However, it does so on the fly. Please see this blog post for details: https://blog.biodiversitylibrary.org/2019/06/bhl-adds-article-download-feature.html. Unpaywall seems willing to work with us since they offered to “add a special case” for BHL content so let’s see how that pans out.

From: Roderic Page notifications@github.com Sent: Wednesday, July 10, 2019 12:24 AM To: rdmpage/australia australia@noreply.github.com Cc: Crowley, Bianca CrowleyB@si.edu; Mention mention@noreply.github.com Subject: Re: [rdmpage/australia] Get Unpaywall working with BHL (#1)

Ah, the comment by @richard-orrhttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Frichard-orr&data=02%7C01%7Ccrowleyb%40si.edu%7Cdaf13304967d4c450dac08d704ee6218%7C989b5e2a14e44efe93b78cdd5fc5d11c%7C0%7C0%7C636983294212824063&sdata=bVoCp6Twcaf6YnQkz7Q5dLVolmOkssJ1aZACFF1o%2B%2Bs%3D&reserved=0 makes sense, I'm assuming that Unpaywall looks for things such as the citation_pdf_url tag to locate the PDF. BHL pages don't include this tag (or anything else machine readable that links to a PDF). Given that (most?) readers just want to read the PDF it would ultimately be useful if BHL had pre-generated articles reading to deliver to the reader. I started some work on this by generating PDFs for BioStor articles and storing them on Internet Archive. Maybe @crowleybhttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fcrowleyb&data=02%7C01%7Ccrowleyb%40si.edu%7Cdaf13304967d4c450dac08d704ee6218%7C989b5e2a14e44efe93b78cdd5fc5d11c%7C0%7C0%7C636983294212824063&sdata=cms2Pq4KRrXrCRmBWvqnGNBeKl93AgKLvwRzBs5fmZU%3D&reserved=0 can comment on whether BHL can generate all article PDFs so that, from a user's perspective, BHL gives them what they most likely want.

@nicolekearneyhttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnicolekearney&data=02%7C01%7Ccrowleyb%40si.edu%7Cdaf13304967d4c450dac08d704ee6218%7C989b5e2a14e44efe93b78cdd5fc5d11c%7C0%7C0%7C636983294212834049&sdata=M7napt5t%2FCYFnPObKflB%2FOe%2FMre7mU6RuVc2thVRw7M%3D&reserved=0 as an aside I think this is another reason to add Google Scholar tags to the Memoirs pages, because at the moment those pages lack the Unpaywall lock symbol.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Frdmpage%2Faustralia%2Fissues%2F1%3Femail_source%3Dnotifications%26email_token%3DAC47PTI6WUNH3ZUMK5ODT2LP6VP4VA5CNFSM4HXWCTSKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZSHYCY%23issuecomment-509901835&data=02%7C01%7Ccrowleyb%40si.edu%7Cdaf13304967d4c450dac08d704ee6218%7C989b5e2a14e44efe93b78cdd5fc5d11c%7C0%7C0%7C636983294212834049&sdata=TaqgjxxDrHKmo%2F0Qd6Qt%2FKIPTAK4LmGDnpSMrb6YOAg%3D&reserved=0, or mute the threadhttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAC47PTLJLTSOQDBXC7ILDB3P6VP4VANCNFSM4HXWCTSA&data=02%7C01%7Ccrowleyb%40si.edu%7Cdaf13304967d4c450dac08d704ee6218%7C989b5e2a14e44efe93b78cdd5fc5d11c%7C0%7C0%7C636983294212844046&sdata=07TzjDzFAnk5ONb%2BqiE5mWRPnBrbJA%2B3LxkouYFf1xo%3D&reserved=0.

rdmpage commented 5 years ago

@crowleyb In the short term yes, if Unpaywall are happy to do things differently that's great, but long term I think BHL needs to think of how best to serve users, most of whom will expect a PDF. Despite the numerous and well known deficiencies of PDFs, they are still what users want.

crowleyb commented 5 years ago

Hi Rod, I’m afraid I don’t understand. BHL is providing users access to an article PDF. If you have a specific request for something different in BHL regarding PDFs please let me know so that I can forward it onto our Technical Team for consideration. The clearer you can be about use cases and expected functionality the better. Thank you for considering the long term of BHL.

From: Roderic Page notifications@github.com Reply-To: rdmpage/australia reply@reply.github.com Date: Wednesday, July 10, 2019 at 18:56 To: rdmpage/australia australia@noreply.github.com Cc: "Crowley, Bianca" CrowleyB@si.edu, Mention mention@noreply.github.com Subject: Re: [rdmpage/australia] Get Unpaywall working with BHL (#1)

@crowleybhttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fcrowleyb&data=02%7C01%7Ccrowleyb%40si.edu%7C9c5d33b149844e85528f08d70589d2da%7C989b5e2a14e44efe93b78cdd5fc5d11c%7C0%7C0%7C636983961827757762&sdata=%2FNUwi8RG2IOPn%2BdGiUoP7hvGQ8t2ndXoGn9SPj%2Bbydc%3D&reserved=0 In the short term yes, if Unpaywall are happy to do things differently that's great, but long term I think BHL needs to think of how best to serve users, most of whom will expect a PDF. Despite the numerous and well known deficiencies of PDFs, they are still what users want.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Frdmpage%2Faustralia%2Fissues%2F1%3Femail_source%3Dnotifications%26email_token%3DAC47PTIMDFWQEDVRA34TYW3P6ZSJHA5CNFSM4HXWCTSKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZU7PLA%23issuecomment-510261164&data=02%7C01%7Ccrowleyb%40si.edu%7C9c5d33b149844e85528f08d70589d2da%7C989b5e2a14e44efe93b78cdd5fc5d11c%7C0%7C0%7C636983961827767751&sdata=2UGKCj0zbWR%2FJ1FKdzWPOfc1sUpbQjwrdKr6MDQm1Y4%3D&reserved=0, or mute the threadhttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAC47PTP5KQ4JF2MQN7U22QDP6ZSJHANCNFSM4HXWCTSA&data=02%7C01%7Ccrowleyb%40si.edu%7C9c5d33b149844e85528f08d70589d2da%7C989b5e2a14e44efe93b78cdd5fc5d11c%7C0%7C0%7C636983961827767751&sdata=yYBsMb6NmuNJtG7HiB%2FUs%2B5Bkes1te6n4ddjddyco4M%3D&reserved=0.

rdmpage commented 5 years ago

@crowleyb The issue is that the way BHL does this is SO clunky. A modern journal provides one click access to the PDF from an article page. Unpaywall even provides one click access to the PDF from the article page ON ANOTHER WEBSITE! This is convenient for people who simply want to read the article right away. Manually selecting pages and waiting for a link to be emailed is crazy in this day and age. I think if we were focussed on users rather than process, BHL would:

generate all article PDFs as articles are identified
embed metadata such as DOIs using, e.g. XMP so reference managers can identify the article
Store the PDFs, e.g. on Internet Archive
provide one-click access to these PDFs

I started doing this with BioStor, but didn’t finish as I simply had too much other stuff to do. But it’s straightforward to automate.

The bigger issue here is thinking about users, and trying to make their reading experience as seamless as a that offered by a modern journal publisher. I suspect this will require a bit of a culture shift in BHL, and the current web interface isn’t set up to do this, but I think BHL is making their users’ life harder than it has to be.

nicolekearney commented 5 years ago

Hi Bianca and Martin,

To answer your question Bianca, BHL does not currently have PDFs of articles available in a way that makes them discoverable by Unpaywall (or Google Scholar). Yes, users can generate them via the BHL website, and this is great, but we really need to have the PDFs pre-generated and linked to the landing pages in order for them to be discoverable.

Having spoken to Rod about this at length today, I really think this is something we should try to do. It will not only make BHL content discoverable via Unpaywall (which is so critical), it will also make it discoverable via Google Scholar (which I would argue is equally critical).

For example, if you copy this article title into Google Scholarhttps://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=%22Description+of+two+new+Species+of+Didelphis+from+Van+Diemen%27s+Land%22&btnG=, you won’t find the BHL versionhttps://www.biodiversitylibrary.org/part/5582#/summary: "Description of two new Species of Didelphis from Van Diemen's Land" (the first description of the Thylacine, published in 1808). Only the version on Wileyhttps://onlinelibrary.wiley.com/doi/abs/10.1111/j.1096-3642.1818.tb00336.x comes up – and that version is behind a paywall!

So, Unpaywall might have suggested that they might incorporate a work around for the fact that BHL doesn’t have PDFs linked from the landing pages of our articles, but I can’t see Google Scholar doing this. We’re going to have to format our content the way every other publisher does if we want them to be able to find it.

The steps we’d need to undertake are (and here I’m rephrasing what Rod has said below):

To pre-generate PDFs for all segments with DOIs
To create a repository for (a collection of) these PDFs either within BHL’s collection on IA (or have them sit somewhere within BHL)
To add a new field on the segment landing page for the links to these PDFs (this could be displayed as a big shiny “PDF” button). This would be in addition to the existing “View article” link that takes you the article start page in each volume.
To add the “citation_pdf_url” meta tag on the segment landing pages. This is what Unpaywall and Google Scholar are looking for.

Rod has produced code for this for BioStor (it’s on Github) so we can perhaps pick his brains about how we could do this for BHL.

It would be awesome if we could consider doing this…

Cheers, Nicole

This e-mail is solely for the named addressee and may be confidential. You should only read, disclose, transmit, copy, distribute, act in reliance on or commercialise the contents if you are authorised to do so. If you are not the intended recipient of this e-mail, please notify postmaster@museum.vic.gov.au mailto:postmaster@museum.vic.gov.au by email immediately, or notify the sender and then destroy any copy of this message. Views expressed in this email are those of the individual sender, except where specifically stated to be those of an officer of Museums Victoria. Museums Victoria does not represent, warrant or guarantee that the integrity of this communication has been maintained nor that it is free from errors, virus or interference.

From: Roderic Page notifications@github.com Reply-To: rdmpage/australia reply@reply.github.com Date: Friday, 12 July 2019 at 9:54 am To: rdmpage/australia australia@noreply.github.com Cc: Nicole Kearney nkearney@museum.vic.gov.au, Mention mention@noreply.github.com Subject: Re: [rdmpage/australia] Get Unpaywall working with BHL (#1)

@crowleybhttps://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fgithub.com%2fcrowleyb&umid=60361f75-076b-484e-bcf5-2975fa1b7561&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-f8de145df7a7374f256d94eeace47903b7691f2b The issue is that the way BHL does this is SO clunky. A modern journal provides one click access to the PDF from an article page. Unpaywall even provides one click access to the PDF from the article page ON ANOTHER WEBSITE! This is convenient for people who simply want to read the article right away. Manually selecting pages and waiting for a link to be emailed is crazy in this day and age. I think if we were focussed on users rather than process, BHL would:

generate all article PDFs as articles are identified
embed metadata such as DOIs using, e.g. XMP so reference managers can identify the article
Store the PDFs, e.g. on Internet Archive
provide one-click access to these PDFs

I started doing this with BioStor, but didn’t finish as I simply had too much other stuff to do. But it’s straightforward to automate.

The bigger issue here is thinking about users, and trying to make their reading experience as seamless as a that offered by a modern journal publisher. I suspect this will require a bit of a culture shift in BHL, and the current web interface isn’t set up to do this, but I think BHL is making their users’ life harder than it has to be.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fgithub.com%2frdmpage%2faustralia%2fissues%2f1%3femail%5fsource%3dnotifications%26email%5ftoken%3dAHQBYGX5D3OFG74AUGFUL73P67B4DA5CNFSM4HXWCTSKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZYJIII%23issuecomment%2d510694433&umid=60361f75-076b-484e-bcf5-2975fa1b7561&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-1ed0e075e3784b953b50f40e0c070256faaa09d6, or mute the threadhttps://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fgithub.com%2fnotifications%2funsubscribe%2dauth%2fAHQBYGXWS66SWWWFF65HGBTP67B4DANCNFSM4HXWCTSA&umid=60361f75-076b-484e-bcf5-2975fa1b7561&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-8f96721e1e0d2b3e2a1d046d38916d7900d84f05.

nicolekearney commented 4 years ago

Hi Nicole, We've finally fixed the issue and we're now correctly linking BHL pages without PDFs: https://api.unpaywall.org/v2/10.1111/j.1096-3642.1818.tb00336.x?email=richard@impactstory.org. About 43k new pages with public domain or CC license info were linked to DOIs. Thanks again for your patience. Richard Orr Lead Developer, Unpaywall OurResearch: We build tools to make scholarly research more open, connected, and reusable—for everyone. https://support.unpaywall.org/public/tickets/130addea2170edd9e4fc198708090fbfc759e801fa0bc184f45c8828c721b636

          On
          Tue, 9 Jul at  4:30 PM
          ,  Crowley, Bianca <crowleyb@si.edu>  wrote:

Dear Richard,

Thank you for getting back to us and explaining the issue. I can confirm that all of BHL content is available for free and open access and as such the “View Article” link will always lead to the full text of the content. Please add a special case for all BHL pages as you suggest below.

We would very much appreciate hearing from you once this has been completed.

Thank you again for taking the time to review our content, eliminate the duplicate entry, and granting us an exception to the rule. Please let me know if you run into any further issues with BHL content and I will do my best to assist.

Regards,

Bianca

Bianca Crowley

Digital Collections Manager

Digital Programs & Initiatives Division

crowleyb@si.edu | 202.633.2239

From: Richard Orr support@unpaywall.org Reply-To: Richard Orr support@unpaywall.org Date: Tuesday, July 9, 2019 at 17:16 To: Nicole Kearney nkearney@museum.vic.gov.au Cc: "reply@reply.github.com" reply@reply.github.com, "Crowley, Bianca" CrowleyB@si.edu Subject: Re: Re: [rdmpage/australia] Get Unpaywall working with BHL (#1)

Hi Nicole,

Thank you for your patience and I'm sorry for not replying sooner. I've been waiting until the problem is solved, which it isn't yet, but I do want to at least let you know what the problem is and that we haven't forgotten about it.

Out problem with the example you provided, and others like

https://www.biodiversitylibrary.org/part/281352, is that on repository pages we look for a PDF link to confirm that the document is actually available. We're not able to recognize the embedded reader on those pages, so they don't get used.

Does the "View article" link always lead to a full copy? If so, I think we can just add a special case for these pages.

The duplicate endpoint doesn't create a problem aside from some wasted harvesting effort on our end, but I'll go ahead and remove one.

https://unpaywall.org/sources/repository/q6caunfjwsunh6wiqwpg will be the one we keep.

Thanks,

Richard Orr

Lead Developer, Unpaywall

Impactstory: We make tools to power the Open Science revolution

https://support.unpaywall.org/public/tickets/130addea2170edd9e4fc198708090fbfc759e801fa0bc184f45c8828c721b636

On Sun, 30 Jun at 6:00 PM , Nicole Kearney nkearney@museum.vic.gov.au wrote:

Dear Richard, Jason and Heather,

Is it possible for you to answer the following questions (from my email below) so we at least know where we’re at:

Is there a reason that the Unpaywall extension still can’t find an open-access version of this article

https://doi.org/10.1111/j.1096-3642.1818.tb00336.x (behind a paywall on Wiley), even though there is an open access version on BHLhttps://www.biodiversitylibrary.org/part/5582#/summary (this is just one example of so many).

Knowing that the BHL content was originally registered with Unpaywall in April, it’s surely not just a matter of just checking every day in the hope that one day it might start working. Or does it really take this long?
Are you able to look into why it isn’t working?

Thank you again for your time, Nicole

Nicole Kearney Manager, Biodiversity Heritage Library Australia Digital Life, Museums Victoria PO Box 666, Melbourne VIC 3001 61 3 8341 7779 biodiversitylibrary.org/collection/bhlauhttps://www.biodiversitylibrary.org/collection/bhlau

This e-mail is solely for the named addressee and may be confidential. You should only read, disclose, transmit, copy, distribute, act in reliance on or commercialise the contents if you are authorised to do so. If you are not the intended recipient of this e-mail, please notify postmaster@museum.vic.gov.au mailto:postmaster@museum.vic.gov.au by email immediately, or notify the sender and then destroy any copy of this message. Views expressed in this email are those of the individual sender, except where specifically stated to be those of an officer of Museums Victoria. Museums Victoria does not represent, warrant or guarantee that the integrity of this communication has been maintained nor that it is free from errors, virus or interference.

From: Nicole Kearney nkearney@museum.vic.gov.au Date: Wednesday, 26 June 2019 at 11:07 am To: Richard Orr support@unpaywall.org, Jason Priem jason@impactstory.org, Heather Piwowar heather@impactstory.org Cc: rdmpage/australia reply@reply.github.com, rdmpage/australia australia@noreply.github.com, Bianca Crowley CrowleyB@si.edu Subject: Re: [rdmpage/australia] Get Unpaywall working with BHL (#1)

Dear Heather, Jason and Richard,

We have been eagerly awaiting the matching of the DOIs in BHL via Unpaywall. I keep checking the Thylacine example:

https://doi.org/10.1111/j.1096-3642.1818.tb00336.x (behind a paywall on Wiley, open access on BHLhttps://www.biodiversitylibrary.org/part/5582#/summary: Unpaywall is still not picking up the open access version in BHL from the Wiley website).

In the meantime we have received the message below from Bianca Crowley at BHL, who informed us that BHL was originally registered with Unpaywall in April 2019 and that by the 15 April, Unpaywall had matched 56,000+ DOIs with a freely accessible version in BHL (we were unaware of this when we reregistered BHL this month). The new endpoint we have been trackinghttps://unpaywall.org/sources/repository/q6caunfjwsunh6wiqwpg now has 95,000+ matches.

We are concerned that, if the BHL DOIs have been matched since April, why is the open access content still not being picked up by Unpaywall? We’re also wondering if it is a problem that there are now two instances of BHL in Unpaywall with two different endpoints: http://www.biodiversitylibrary.org/oai

https://www.biodiversitylibrary.org/oai

We’re extremely keen to see the BHL content discoverable via Unpaywall as soon as possible and we greatly appreciate your time trying to resolve this. Please let us know if there is anything we can do to help.

Kind regards, Nicole (and Rod)

From: Bianca Crowley notifications@github.com Reply-To: rdmpage/australia reply@reply.github.com Date: Wednesday, 26 June 2019 at 4:23 am To: rdmpage/australia australia@noreply.github.com Cc: Nicole Kearney nkearney@museum.vic.gov.au, Mention mention@noreply.github.com Subject: Re: [rdmpage/australia] Get Unpaywall working with BHL (#1)

BHL originally registered with Unpaywall on April 3, 2019. As of April 15 our status was as follows: [image]https://hes32-ctp.trendmicro.com/wis/clicktime/v1/query?url=https%3a%2f%2fuser%2dimages.githubusercontent.com%2f12187597%2f60123042%2d866f5100%2d9754%2d11e9%2d9531%2d9d3934638c96.png&umid=db449783-476b-4de4-a58b-637002129e4e&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-f516eeea6debf27733857599699bbfc7ac25989a But now it is different: https://unpaywall.org/sources/repository/fsxfk6gcvszjgobj4jnt https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2funpaywall.org%2fsources%2frepository%2ffsxfk6gcvszjgobj4jnt&umid=db449783-476b-4de4-a58b-637002129e4e&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-651b91e3a30411dd09230b402fe71f5e260669d2

1https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fgithub.com%2frdmpage%2faustralia%2fissues%2f1&umid=db449783-476b-4de4-a58b-637002129e4e&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-c9cef49640d5dc2b77d0c2ca2082573dd1e891f5

We'll need to see what happened with the first batch that was registered and why it changed.

2https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fgithub.com%2frdmpage%2faustralia%2fissues%2f2&umid=db449783-476b-4de4-a58b-637002129e4e&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-9036f11bfa8b91c9b8aca6725f62a785d8e89634

it will also be useful now to make sure there is only one instance of BHL represented in Unpaywall

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fgithub.com%2frdmpage%2faustralia%2fissues%2f1%3femail%5fsource%3dnotifications%26email%5ftoken%3dAHQBYGSNSNO7QC27PBEA2BDP4JPBXA5CNFSM4HXWCTSKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYREYEY%23issuecomment%2d505564179&umid=db449783-476b-4de4-a58b-637002129e4e&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-f956f4a13128536e57c4fdc4b07dd1d7345d2e96, or mute the threadhttps://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fgithub.com%2fnotifications%2funsubscribe%2dauth%2fAHQBYGXDFZUTYLLPAM37JN3P4JPBXANCNFSM4HXWCTSA&umid=db449783-476b-4de4-a58b-637002129e4e&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-3d518f083ee7b086b7a2249eac2d914467c4b457.

nicolekearney commented 4 years ago

Dear Richard,

Thanks again for making it possible for BHL content to be discoverable via your Paywall extension. We’re all still buzzing about it (in fact there will be a blog post published about it on the BHL blog today).

We would very much like to track how much traffic comes to BHL as a result of the Unpaywall fix. However, it seems that BHL is unable to recognise Unpaywall as the source of web traffic; we believe the referrals will be reported as coming from the publisher/etc. sites themselves.

I assume you must have come up against this before. Do you have a way of getting around this? Do you set headers when sending a user from a publisher’s site to an open access article? Can we use these to track the Unpaywall traffic to BHL?

I’d really like to gather these stats and to (hopefully) use them to justify why we need more articles, more article-level metadata and more DOIs to BHL (so that even more of our content can be discoverable via Unpaywall).

Any direction you could give me would be greatly appreciated, Nicole

P.S. I was also wondering whether you might consider adding the BHL to your list of logos next to “Used and trusted by top organizations” on your homepage. ☺

Nicole Kearney Manager, Biodiversity Heritage Library Australia Digital Life, Museums Victoria PO Box 666, Melbourne VIC 3001 61 3 8341 7779 biodiversitylibrary.org/collection/bhlauhttps://www.biodiversitylibrary.org/collection/bhlau

From: Richard Orr support@unpaywall.org Sent: Tuesday, 13 August 2019 12:22 PM To: Nicole Kearney nkearney@museum.vic.gov.au Cc: reply@reply.github.com; crowleyb@si.edu Subject: Re: Re: [rdmpage/australia] Get Unpaywall working with BHL (#1)

Hi Nicole,

We've finally fixed the issue and we're now correctly linking BHL pages without PDFs: https://api.unpaywall.org/v2/10.1111/j.1096-3642.1818.tb00336.x?email=richard@impactstory.org https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fapi.unpaywall.org%2fv2%2f10.1111%2fj.1096%2d3642.1818.tb00336.x%3femail%3drichard%40impactstory.org&umid=fb685a6f-6973-4de3-8e2f-b4e796513ab4&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-f812c7d92cb6266b50e866d82beb600c44a207b7. About 43k new pages with public domain or CC license info were linked to DOIs. Thanks again for your patience.

Richard Orr Lead Developer, Unpaywallhttps://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=http%3a%2f%2funpaywall.org&umid=fb685a6f-6973-4de3-8e2f-b4e796513ab4&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-fbda7ea322124c8b286ed4c5c8db39b9abd7ad92 OurResearchhttps://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2four%2dresearch.org&umid=fb685a6f-6973-4de3-8e2f-b4e796513ab4&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-53234f4c968cfbc90c5e493f527a345768d388d6: We build tools to make scholarly research more open, connected, and reusable—for everyone.

https://support.unpaywall.org/public/tickets/130addea2170edd9e4fc198708090fbfc759e801fa0bc184f45c8828c721b636 https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fsupport.unpaywall.org%2fpublic%2ftickets%2f130addea2170edd9e4fc198708090fbfc759e801fa0bc184f45c8828c721b636%c2%a0&umid=fb685a6f-6973-4de3-8e2f-b4e796513ab4&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-9cfb3a179e914a63608228430828c070942f13cb On Tue, 9 Jul at 4:30 PM , Crowley, Bianca crowleyb@si.edu wrote: Dear Richard,

Thank you for getting back to us and explaining the issue. I can confirm that all of BHL content is available for free and open access and as such the “View Article” link will always lead to the full text of the content. Please add a special case for all BHL pages as you suggest below.

We would very much appreciate hearing from you once this has been completed.

Thank you again for taking the time to review our content, eliminate the duplicate entry, and granting us an exception to the rule. Please let me know if you run into any further issues with BHL content and I will do my best to assist.

Regards, Bianca

Bianca Crowley Digital Collections Manager Digital Programs & Initiatives Division crowleyb@si.edumailto:crowleyb@si.edu | 202.633.2239

[Image removed by sender. EmailSignature_option1_noTag_RGB]

From: Richard Orr support@unpaywall.org Reply-To: Richard Orr support@unpaywall.org Date: Tuesday, July 9, 2019 at 17:16 To: Nicole Kearney nkearney@museum.vic.gov.au Cc: "reply@reply.github.com" reply@reply.github.com, "Crowley, Bianca" CrowleyB@si.edu Subject: Re: Re: [rdmpage/australia] Get Unpaywall working with BHL (#1)

Hi Nicole,

Thank you for your patience and I'm sorry for not replying sooner. I've been waiting until the problem is solved, which it isn't yet, but I do want to at least let you know what the problem is and that we haven't forgotten about it.

Out problem with the example you provided, and others like https://www.biodiversitylibrary.org/part/281352,https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fnam02.safelinks.protection.outlook.com%2f%3furl%3dhttps%253A%252F%252Fwww.biodiversitylibrary.org%252Fpart%252F281352%252C%26data%3d02%257C01%257Ccrowleyb%2540si.edu%257C9bb8b83952f24a34243e08d704b2b88d%257C989b5e2a14e44efe93b78cdd5fc5d11c%257C0%257C0%257C636983037995244991%26sdata%3d%252BG6%252FyzNjbrlAyvxfMPEOLkTb7nOkVv0us0Kd6vBH3bM%253D%26reserved%3d0&umid=fb685a6f-6973-4de3-8e2f-b4e796513ab4&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-85c859e3d1fc3997735962782bafbeb8bfa649b2 is that on repository pages we look for a PDF link to confirm that the document is actually available. We're not able to recognize the embedded reader on those pages, so they don't get used.

Does the "View article" link always lead to a full copy? If so, I think we can just add a special case for these pages.

The duplicate endpoint doesn't create a problem aside from some wasted harvesting effort on our end, but I'll go ahead and remove one. https://unpaywall.org/sources/repository/q6caunfjwsunh6wiqwpg https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fnam02.safelinks.protection.outlook.com%2f%3furl%3dhttps%253A%252F%252Funpaywall.org%252Fsources%252Frepository%252Fq6caunfjwsunh6wiqwpg%26data%3d02%257C01%257Ccrowleyb%2540si.edu%257C9bb8b83952f24a34243e08d704b2b88d%257C989b5e2a14e44efe93b78cdd5fc5d11c%257C0%257C1%257C636983037995244991%26sdata%3dMSS3749mAbL07IW9zHqfbxGQnWUsvPnpXoiMzRN9pmM%253D%26reserved%3d0&umid=fb685a6f-6973-4de3-8e2f-b4e796513ab4&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-033eca5d498bb716bcbd9608218a208577f75440 will be the one we keep.

Thanks,

Richard Orr Lead Developer, Unpaywall Impactstoryhttps://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fnam02.safelinks.protection.outlook.com%2f%3furl%3dhttp%253A%252F%252Fimpactstory.org%252F%26data%3d02%257C01%257Ccrowleyb%2540si.edu%257C9bb8b83952f24a34243e08d704b2b88d%257C989b5e2a14e44efe93b78cdd5fc5d11c%257C0%257C1%257C636983037995254982%26sdata%3diWaG5uL1AtKnIEqc4LQEIkLzQxDHtz7D8Mvir8qIiRA%253D%26reserved%3d0&umid=fb685a6f-6973-4de3-8e2f-b4e796513ab4&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-5fc612e6098377485469c76a167fa19fc112ed2d: We make tools to power the Open Science revolution

https://support.unpaywall.org/public/tickets/130addea2170edd9e4fc198708090fbfc759e801fa0bc184f45c8828c721b636 https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fnam02.safelinks.protection.outlook.com%2f%3furl%3dhttps%253A%252F%252Fsupport.unpaywall.org%252Fpublic%252Ftickets%252F130addea2170edd9e4fc198708090fbfc759e801fa0bc184f45c8828c721b636%26data%3d02%257C01%257Ccrowleyb%2540si.edu%257C9bb8b83952f24a34243e08d704b2b88d%257C989b5e2a14e44efe93b78cdd5fc5d11c%257C0%257C1%257C636983037995254982%26sdata%3dqbJLBuBxFro8yCl4pzukWJUrJ3J2%252F6LBMhgO8YCYP1s%253D%26reserved%3d0&umid=fb685a6f-6973-4de3-8e2f-b4e796513ab4&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-6a4ef169840846748f208bee4d0dc42b4a1d7093 On Sun, 30 Jun at 6:00 PM , Nicole Kearney nkearney@museum.vic.gov.au wrote: Dear Richard, Jason and Heather,

Is it possible for you to answer the following questions (from my email below) so we at least know where we’re at:

Is there a reason that the Unpaywall extension still can’t find an open-access version of this article https://doi.org/10.1111/j.1096-3642.1818.tb00336.x https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fnam02.safelinks.protection.outlook.com%2f%3furl%3dhttps%253A%252F%252Fdoi.org%252F10.1111%252Fj.1096%2d3642.1818.tb00336.x%26data%3d02%257C01%257Ccrowleyb%2540si.edu%257C9bb8b83952f24a34243e08d704b2b88d%257C989b5e2a14e44efe93b78cdd5fc5d11c%257C0%257C1%257C636983037995264977%26sdata%3dlo1q%252FEW6XQE2f%252FTdqsBrSBocsu3SL5JrpKkE7O4%252B3DU%253D%26reserved%3d0&umid=fb685a6f-6973-4de3-8e2f-b4e796513ab4&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-bce16a35031f797c1b19e217f6e8011cb75f7fc1 (behind a paywall on Wiley), even though there is an open access version on BHLhttps://www.biodiversitylibrary.org/part/5582#/summary<https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fnam02.safelinks.protection.outlook.com%2f%3furl%3dhttps%253A%252F%252Fwww.biodiversitylibrary.org%252Fpart%252F5582%2523%252Fsummary%26data%3d02%257C01%257Ccrowleyb%2540si.edu%257C9bb8b83952f24a34243e08d704b2b88d%257C989b5e2a14e44efe93b78cdd5fc5d11c%257C0%257C1%257C636983037995264977%26sdata%3d2NN%252FBJkM5qYshQi4sqRs%252FivJc6Wb22HL00UU4F4iCfE%253D%26reserved%3d0&umid=fb685a6f-6973-4de3-8e2f-b4e796513ab4&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-64519eae6d387bed1209d57da4868f07e59fb385> (this is just one example of so many).
Knowing that the BHL content was originally registered with Unpaywall in April, it’s surely not just a matter of just checking every day in the hope that one day it might start working. Or does it really take this long?
Are you able to look into why it isn’t working?

Thank you again for your time, Nicole

Nicole Kearney Manager, Biodiversity Heritage Library Australia Digital Life, Museums Victoria PO Box 666, Melbourne VIC 3001 61 3 8341 7779 biodiversitylibrary.org/collection/bhlauhttps://www.biodiversitylibrary.org/collection/bhlau<https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fnam02.safelinks.protection.outlook.com%2f%3furl%3dhttps%253A%252F%252Fwww.biodiversitylibrary.org%252Fcollection%252Fbhlau%26data%3d02%257C01%257Ccrowleyb%2540si.edu%257C9bb8b83952f24a34243e08d704b2b88d%257C989b5e2a14e44efe93b78cdd5fc5d11c%257C0%257C0%257C636983037995274970%26sdata%3dnRNQ8vehdggMexTxlc85k8HFSBXuVIKA6rlEwMmbqjY%253D%26reserved%3d0&umid=fb685a6f-6973-4de3-8e2f-b4e796513ab4&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-3669114013ff7cd7aecc39948f22cf9ea3a1724d>

This e-mail is solely for the named addressee and may be confidential. You should only read, disclose, transmit, copy, distribute, act in reliance on or commercialise the contents if you are authorised to do so. If you are not the intended recipient of this e-mail, please notify postmaster@museum.vic.gov.au mailto:postmaster@museum.vic.gov.au by email immediately, or notify the sender and then destroy any copy of this message. Views expressed in this email are those of the individual sender, except where specifically stated to be those of an officer of Museums Victoria. Museums Victoria does not represent, warrant or guarantee that the integrity of this communication has been maintained nor that it is free from errors, virus or interference. From: Nicole Kearney nkearney@museum.vic.gov.au Date: Wednesday, 26 June 2019 at 11:07 am To: Richard Orr support@unpaywall.org, Jason Priem jason@impactstory.org, Heather Piwowar heather@impactstory.org Cc: rdmpage/australia reply@reply.github.com, rdmpage/australia australia@noreply.github.com, Bianca Crowley CrowleyB@si.edu Subject: Re: [rdmpage/australia] Get Unpaywall working with BHL (#1)

Dear Heather, Jason and Richard,

We have been eagerly awaiting the matching of the DOIs in BHL via Unpaywall. I keep checking the Thylacine example: https://doi.org/10.1111/j.1096-3642.1818.tb00336.x https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fnam02.safelinks.protection.outlook.com%2f%3furl%3dhttps%253A%252F%252Fdoi.org%252F10.1111%252Fj.1096%2d3642.1818.tb00336.x%26data%3d02%257C01%257Ccrowleyb%2540si.edu%257C9bb8b83952f24a34243e08d704b2b88d%257C989b5e2a14e44efe93b78cdd5fc5d11c%257C0%257C1%257C636983037995274970%26sdata%3dU6foh8SzhkToapIa3sS9XZyY%252BFpwR5C7TCnne%252BxHyEU%253D%26reserved%3d0&umid=fb685a6f-6973-4de3-8e2f-b4e796513ab4&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-6981286fdc0056c015d09c7078e35f8712ab3f73 (behind a paywall on Wiley, open access on BHLhttps://www.biodiversitylibrary.org/part/5582#/summary:https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fnam02.safelinks.protection.outlook.com%2f%3furl%3dhttps%253A%252F%252Fwww.biodiversitylibrary.org%252Fpart%252F5582%2523%252Fsummary%253E%253A%26data%3d02%257C01%257Ccrowleyb%2540si.edu%257C9bb8b83952f24a34243e08d704b2b88d%257C989b5e2a14e44efe93b78cdd5fc5d11c%257C0%257C0%257C636983037995274970%26sdata%3dhf6eZcKfZD1zbEp4c2w8QHH1dc8k%252B3pkLpZVY62jFlc%253D%26reserved%3d0&umid=fb685a6f-6973-4de3-8e2f-b4e796513ab4&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-6c39cc5ac40f94f51d5ef6953ae7bd7d1f8644c6 Unpaywall is still not picking up the open access version in BHL from the Wiley website).

In the meantime we have received the message below from Bianca Crowley at BHL, who informed us that BHL was originally registered with Unpaywall in April 2019 and that by the 15 April, Unpaywall had matched 56,000+ DOIs with a freely accessible version in BHL (we were unaware of this when we reregistered BHL this month). The new endpoint we have been trackinghttps://unpaywall.org/sources/repository/q6caunfjwsunh6wiqwpg<https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fnam02.safelinks.protection.outlook.com%2f%3furl%3dhttps%253A%252F%252Funpaywall.org%252Fsources%252Frepository%252Fq6caunfjwsunh6wiqwpg%26data%3d02%257C01%257Ccrowleyb%2540si.edu%257C9bb8b83952f24a34243e08d704b2b88d%257C989b5e2a14e44efe93b78cdd5fc5d11c%257C0%257C1%257C636983037995284964%26sdata%3dZr9mWPMBqZBejWCxyW1tjaS8orQ4JGmqRlTn8UUVqt4%253D%26reserved%3d0&umid=fb685a6f-6973-4de3-8e2f-b4e796513ab4&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-a7e5fb02b9355de398973541d02bffa3965ac08d> now has 95,000+ matches.

We are concerned that, if the BHL DOIs have been matched since April, why is the open access content still not being picked up by Unpaywall? We’re also wondering if it is a problem that there are now two instances of BHL in Unpaywall with two different endpoints: http://www.biodiversitylibrary.org/oai https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fnam02.safelinks.protection.outlook.com%2f%3furl%3dhttp%253A%252F%252Fwww.biodiversitylibrary.org%252Foai%26data%3d02%257C01%257Ccrowleyb%2540si.edu%257C9bb8b83952f24a34243e08d704b2b88d%257C989b5e2a14e44efe93b78cdd5fc5d11c%257C0%257C0%257C636983037995284964%26sdata%3dd3tY0Q9ZT05BMjJN%252B6l15insra6y7mcCU%252B81RJk%252FJN8%253D%26reserved%3d0&umid=fb685a6f-6973-4de3-8e2f-b4e796513ab4&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-e04a17d9f439acb65e2a8fda0713c7623253a9b6

https://www.biodiversitylibrary.org/oai https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fnam02.safelinks.protection.outlook.com%2f%3furl%3dhttps%253A%252F%252Fwww.biodiversitylibrary.org%252Foai%26data%3d02%257C01%257Ccrowleyb%2540si.edu%257C9bb8b83952f24a34243e08d704b2b88d%257C989b5e2a14e44efe93b78cdd5fc5d11c%257C0%257C0%257C636983037995294961%26sdata%3d4rZvpCKdkTmp0lcyIPxsvV6EOW4NNJdet%252FUzZ%252F2eYDg%253D%26reserved%3d0&umid=fb685a6f-6973-4de3-8e2f-b4e796513ab4&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-c792ad70e19f7f77205cbca96c14b32a2d34e4ab

We’re extremely keen to see the BHL content discoverable via Unpaywall as soon as possible and we greatly appreciate your time trying to resolve this. Please let us know if there is anything we can do to help.

Kind regards, Nicole (and Rod)

From: Bianca Crowley notifications@github.com Reply-To: rdmpage/australia reply@reply.github.com Date: Wednesday, 26 June 2019 at 4:23 am To: rdmpage/australia australia@noreply.github.com Cc: Nicole Kearney nkearney@museum.vic.gov.au, Mention mention@noreply.github.com Subject: Re: [rdmpage/australia] Get Unpaywall working with BHL (#1)

BHL originally registered with Unpaywall on April 3, 2019. As of April 15 our status was as follows: [image]https://hes32-ctp.trendmicro.com/wis/clicktime/v1/query?url=https%3a%2f%2fuser%2dimages.githubusercontent.com%2f12187597%2f60123042%2d866f5100%2d9754%2d11e9%2d9531%2d9d3934638c96.png&umid=db449783-476b-4de4-a58b-637002129e4e&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-f516eeea6debf27733857599699bbfc7ac25989a<https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fnam02.safelinks.protection.outlook.com%2f%3furl%3dhttps%253A%252F%252Fhes32%2dctp.trendmicro.com%252Fwis%252Fclicktime%252Fv1%252Fquery%253Furl%253Dhttps%25253a%25252f%25252fuser%25252dimages.githubusercontent.com%25252f12187597%25252f60123042%25252d866f5100%25252d9754%25252d11e9%25252d9531%25252d9d3934638c96.png%2526umid%253Ddb449783%2d476b%2d4de4%2da58b%2d637002129e4e%2526auth%253D89a422ce48cf9afc268cabe806cc53ea452e36bd%2df516eeea6debf27733857599699bbfc7ac25989a%26data%3d02%257C01%257Ccrowleyb%2540si.edu%257C9bb8b83952f24a34243e08d704b2b88d%257C989b5e2a14e44efe93b78cdd5fc5d11c%257C0%257C0%257C636983037995294961%26sdata%3dmKgCYprrUxQH17dhiwsL%252FTr1P74AjF2m9ZfurxoZXno%253D%26reserved%3d0&umid=fb685a6f-6973-4de3-8e2f-b4e796513ab4&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-074aadea251fc3f4899a052e32aa13abd5f3f02c> But now it is different: https://unpaywall.org/sources/repository/fsxfk6gcvszjgobj4jnt https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2funpaywall.org%2fsources%2frepository%2ffsxfk6gcvszjgobj4jnt&umid=db449783-476b-4de4-a58b-637002129e4e&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-651b91e3a30411dd09230b402fe71f5e260669d2<https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fnam02.safelinks.protection.outlook.com%2f%3furl%3dhttps%253A%252F%252Funpaywall.org%252Fsources%252Frepository%252Ffsxfk6gcvszjgobj4jnt%253Chttps%253A%252F%252Fhes32%2dctp.trendmicro.com%253A443%252Fwis%252Fclicktime%252Fv1%252Fquery%253Furl%253Dhttps%25253a%25252f%25252funpaywall.org%25252fsources%25252frepository%25252ffsxfk6gcvszjgobj4jnt%2526umid%253Ddb449783%2d476b%2d4de4%2da58b%2d637002129e4e%2526auth%253D89a422ce48cf9afc268cabe806cc53ea452e36bd%2d651b91e3a30411dd09230b402fe71f5e260669d2%26data%3d02%257C01%257Ccrowleyb%2540si.edu%257C9bb8b83952f24a34243e08d704b2b88d%257C989b5e2a14e44efe93b78cdd5fc5d11c%257C0%257C1%257C636983037995304950%26sdata%3dy19ZheIFOb6TLoFA12Yn4xds63hsnNmO2Kb0l8zmR9A%253D%26reserved%3d0&umid=fb685a6f-6973-4de3-8e2f-b4e796513ab4&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-5308ec467e6f11bce1815b3fc7995859f1d2b3b1>

1https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fgithub.com%2frdmpage%2faustralia%2fissues%2f1&umid=db449783-476b-4de4-a58b-637002129e4e&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-c9cef49640d5dc2b77d0c2ca2082573dd1e891f5<https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fnam02.safelinks.protection.outlook.com%2f%3furl%3dhttps%253A%252F%252Fhes32%2dctp.trendmicro.com%253A443%252Fwis%252Fclicktime%252Fv1%252Fquery%253Furl%253Dhttps%25253a%25252f%25252fgithub.com%25252frdmpage%25252faustralia%25252fissues%25252f1%2526umid%253Ddb449783%2d476b%2d4de4%2da58b%2d637002129e4e%2526auth%253D89a422ce48cf9afc268cabe806cc53ea452e36bd%2dc9cef49640d5dc2b77d0c2ca2082573dd1e891f5%26data%3d02%257C01%257Ccrowleyb%2540si.edu%257C9bb8b83952f24a34243e08d704b2b88d%257C989b5e2a14e44efe93b78cdd5fc5d11c%257C0%257C0%257C636983037995304950%26sdata%3dZxb7mHAx2Dm1DhJCwtzxpfb2gHgo1DV2JYe2sKaFdMY%253D%26reserved%3d0&umid=fb685a6f-6973-4de3-8e2f-b4e796513ab4&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-3ffda013e9da028fb4249378e5625058e373e128> We'll need to see what happened with the first batch that was registered and why it changed.

2https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fgithub.com%2frdmpage%2faustralia%2fissues%2f2&umid=db449783-476b-4de4-a58b-637002129e4e&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-9036f11bfa8b91c9b8aca6725f62a785d8e89634<https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fnam02.safelinks.protection.outlook.com%2f%3furl%3dhttps%253A%252F%252Fhes32%2dctp.trendmicro.com%253A443%252Fwis%252Fclicktime%252Fv1%252Fquery%253Furl%253Dhttps%25253a%25252f%25252fgithub.com%25252frdmpage%25252faustralia%25252fissues%25252f2%2526umid%253Ddb449783%2d476b%2d4de4%2da58b%2d637002129e4e%2526auth%253D89a422ce48cf9afc268cabe806cc53ea452e36bd%2d9036f11bfa8b91c9b8aca6725f62a785d8e89634%26data%3d02%257C01%257Ccrowleyb%2540si.edu%257C9bb8b83952f24a34243e08d704b2b88d%257C989b5e2a14e44efe93b78cdd5fc5d11c%257C0%257C0%257C636983037995314947%26sdata%3d2epErYZZCPUxEzIKSmvWHK9%252BInrUetdVuGNwny6sHGo%253D%26reserved%3d0&umid=fb685a6f-6973-4de3-8e2f-b4e796513ab4&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-0ce44cb2dd2db0e0d2fd0344dc14326f0e146d85> it will also be useful now to make sure there is only one instance of BHL represented in Unpaywall

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fgithub.com%2frdmpage%2faustralia%2fissues%2f1%3femail%5fsource%3dnotifications%26email%5ftoken%3dAHQBYGSNSNO7QC27PBEA2BDP4JPBXA5CNFSM4HXWCTSKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYREYEY%23issuecomment%2d505564179&umid=db449783-476b-4de4-a58b-637002129e4e&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-f956f4a13128536e57c4fdc4b07dd1d7345d2e96<https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fnam02.safelinks.protection.outlook.com%2f%3furl%3dhttps%253A%252F%252Fhes32%2dctp.trendmicro.com%253A443%252Fwis%252Fclicktime%252Fv1%252Fquery%253Furl%253Dhttps%25253a%25252f%25252fgithub.com%25252frdmpage%25252faustralia%25252fissues%25252f1%25253femail%25255fsource%25253dnotifications%252526email%25255ftoken%25253dAHQBYGSNSNO7QC27PBEA2BDP4JPBXA5CNFSM4HXWCTSKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYREYEY%252523issuecomment%25252d505564179%2526umid%253Ddb449783%2d476b%2d4de4%2da58b%2d637002129e4e%2526auth%253D89a422ce48cf9afc268cabe806cc53ea452e36bd%2df956f4a13128536e57c4fdc4b07dd1d7345d2e96%26data%3d02%257C01%257Ccrowleyb%2540si.edu%257C9bb8b83952f24a34243e08d704b2b88d%257C989b5e2a14e44efe93b78cdd5fc5d11c%257C0%257C0%257C636983037995324939%26sdata%3dfzH8D8H1ePPYtkC%252FRSDdGAicyOwk4iTLRoUcUc%252B%252BOuo%253D%26reserved%3d0&umid=fb685a6f-6973-4de3-8e2f-b4e796513ab4&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-d1c73223f03eef18857223f212d2b50b814d33b1>, or mute the threadhttps://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fgithub.com%2fnotifications%2funsubscribe%2dauth%2fAHQBYGXDFZUTYLLPAM37JN3P4JPBXANCNFSM4HXWCTSA&umid=db449783-476b-4de4-a58b-637002129e4e&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-3d518f083ee7b086b7a2249eac2d914467c4b457<https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fnam02.safelinks.protection.outlook.com%2f%3furl%3dhttps%253A%252F%252Fhes32%2dctp.trendmicro.com%253A443%252Fwis%252Fclicktime%252Fv1%252Fquery%253Furl%253Dhttps%25253a%25252f%25252fgithub.com%25252fnotifications%25252funsubscribe%25252dauth%25252fAHQBYGXDFZUTYLLPAM37JN3P4JPBXANCNFSM4HXWCTSA%2526umid%253Ddb449783%2d476b%2d4de4%2da58b%2d637002129e4e%2526auth%253D89a422ce48cf9afc268cabe806cc53ea452e36bd%2d3d518f083ee7b086b7a2249eac2d914467c4b457%26data%3d02%257C01%257Ccrowleyb%2540si.edu%257C9bb8b83952f24a34243e08d704b2b88d%257C989b5e2a14e44efe93b78cdd5fc5d11c%257C0%257C0%257C636983037995324939%26sdata%3ducjqrDZ%252BFGc4rgxADRDVZz15Uu%252Ftk%252Beqje5OWyC9lo8%253D%26reserved%3d0&umid=fb685a6f-6973-4de3-8e2f-b4e796513ab4&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-56f46adf6bc7995f336aab2c061066b3da0cb6a4>. 443:1048800

This e-mail is solely for the named addressee and may be confidential. You should only read, disclose, transmit, copy, distribute, act in reliance on or commercialise the contents if you are authorised to do so. If you are not the intended recipient of this e-mail, please notify postmaster@museum.vic.gov.au mailto:postmaster@museum.vic.gov.au by email immediately, or notify the sender and then destroy any copy of this message. Views expressed in this email are those of the individual sender, except where specifically stated to be those of an officer of Museums Victoria. Museums Victoria does not represent, warrant or guarantee that the integrity of this communication has been maintained nor that it is free from errors, virus or interference.

nicolekearney commented 4 years ago

Hi Nicole, At least in Chrome, we generate requests without any referer header, which might be unusual enough that you could attribute most of any increase in such requests to Unpaywall. Technically this is required behavior, but I don't know whether breaking the rules here is a big deal. I'm sorry to say we don't have a lot of bandwidth right now to evaluate it. I'll put this on hold as a feature request. I'm CCing Jason, one of our co-founders, about adding the logo. Richard Orr Lead Developer, Unpaywall OurResearch: We build tools to make scholarly research more open, connected, and reusable—for everyone. https://support.unpaywall.org/public/tickets/130addea2170edd9e4fc198708090fbfc759e801fa0bc184f45c8828c721b636

          On
          Thu, 15 Aug at  9:42 PM
          ,  Nicole Kearney <nkearney@museum.vic.gov.au>  wrote:

Dear Richard,

Thanks again for making it possible for BHL content to be discoverable via your Paywall extension. We’re all still buzzing about it (in fact there will be a blog post published about it on the BHL blog today).

We would very much like to track how much traffic comes to BHL as a result of the Unpaywall fix. However, it seems that BHL is unable to recognise Unpaywall as the source of web traffic; we believe the referrals will be reported as coming from the publisher/etc. sites themselves.

I assume you must have come up against this before. Do you have a way of getting around this? Do you set headers when sending a user from a publisher’s site to an open access article? Can we use these to track the Unpaywall traffic to BHL?

I’d really like to gather these stats and to (hopefully) use them to justify why we need more articles, more article-level metadata and more DOIs to BHL (so that even more of our content can be discoverable via Unpaywall).

Any direction you could give me would be greatly appreciated, Nicole

P.S. I was also wondering whether you might consider adding the BHL to your list of logos next to “Used and trusted by top organizations” on your homepage. J

Nicole Kearney

Manager, Biodiversity Heritage Library Australia

Digital Life, Museums Victoria PO Box 666, Melbourne VIC 3001

61 3 8341 7779

biodiversitylibrary.org/collection/bhlau

From: Richard Orr support@unpaywall.org

Sent: Tuesday, 13 August 2019 12:22 PM To: Nicole Kearney nkearney@museum.vic.gov.au Cc: reply@reply.github.com; crowleyb@si.edu Subject: Re: Re: [rdmpage/australia] Get Unpaywall working with BHL (#1)

Hi Nicole,

We've finally fixed the issue and we're now correctly linking BHL pages without PDFs: https://api.unpaywall.org/v2/10.1111/j.1096-3642.1818.tb00336.x?email=richard@impactstory.org. About 43k new pages with public domain or CC license info were linked to DOIs. Thanks again for your patience.

Richard Orr

Lead Developer, Unpaywall

OurResearch: We build tools to make scholarly research more open, connected,

and reusable—for everyone.

https://support.unpaywall.org/public/tickets/130addea2170edd9e4fc198708090fbfc759e801fa0bc184f45c8828c721b636

On Tue, 9 Jul at 4:30 PM , Crowley, Bianca crowleyb@si.edu wrote:

Dear Richard,

Thank you for getting back to us and explaining the issue. I can confirm that all of BHL content is available for free and open access and as such the “View Article” link will always lead to the full text of the content. Please add a special case for all BHL pages as you suggest below.

We would very much appreciate hearing from you once this has been completed.

Thank you again for taking the time to review our content, eliminate the duplicate entry, and granting us an exception to the rule. Please let me know if you run into any further issues with BHL content and I will do my best to assist.

Regards,

Bianca

Bianca Crowley

Digital Collections Manager

Digital Programs & Initiatives Division

crowleyb@si.edu | 202.633.2239

From: Richard Orr support@unpaywall.org Reply-To: Richard Orr support@unpaywall.org Date: Tuesday, July 9, 2019 at 17:16 To: Nicole Kearney nkearney@museum.vic.gov.au Cc: "reply@reply.github.com" reply@reply.github.com, "Crowley, Bianca" CrowleyB@si.edu Subject: Re: Re: [rdmpage/australia] Get Unpaywall working with BHL (#1)

Hi Nicole,

Thank you for your patience and I'm sorry for not replying sooner. I've been waiting until the problem is solved, which it isn't yet, but I do want to at least let you know what the problem is and that we haven't forgotten about it.

Out problem with the example you provided, and others like https://www.biodiversitylibrary.org/part/281352, is that on repository pages we look for a PDF link to confirm that the document is actually available. We're not able to recognize the embedded reader on those pages, so they don't get used.

Does the "View article" link always lead to a full copy? If so, I think we can just add a special case for these pages.

The duplicate endpoint doesn't create a problem aside from some wasted harvesting effort on our end, but I'll go ahead and remove one. https://unpaywall.org/sources/repository/q6caunfjwsunh6wiqwpg will be the one we keep.

Thanks,

Richard Orr

Lead Developer, Unpaywall

Impactstory: We make tools to power the Open Science revolution

https://support.unpaywall.org/public/tickets/130addea2170edd9e4fc198708090fbfc759e801fa0bc184f45c8828c721b636

This e-mail is solely for the named addressee and may be confidential. You should only read, disclose, transmit, copy, distribute, act in reliance on or commercialise the contents if you are authorised to do so. If you are not the intended recipient of this e-mail, please notify postmaster@museum.vic.gov.au by email immediately, or notify the sender and then destroy any copy of this message. Views expressed in this email are those of the individual sender, except where specifically stated to be those of an officer of Museums Victoria. Museums Victoria does not represent, warrant or guarantee that the integrity of this communication has been maintained nor that it is free from errors, virus or interference.

crowleyb commented 4 years ago

Hi Nicole, Rod,

I was revisiting your message from below to make sure it didn’t get lost in the mix and have forwarded the issue to BHL’s Gemini system for review by the Tech Team when time allows. For future reference, please email feedback@biodiversitylibrary.orgmailto:feedback@biodiversitylibrary.org to submit requests directly to Gemini.

I’m also not sure if Martin actually got the message below (I thought he mentioned it to me but I cannot see his email below). Either way, Gemini is the best place to send these things. You are also welcome to send them onto me and I’ll get them into Gemini for you. Joel Richard is our new BHL Technical Coordinator and he is going through Gemini with more regularity since he’s taken over the role than folks have done in the past.

Please understand I am doing what I can to get your request to the right place for follow up. Let me know if you have any questions or concerns.

Thanks again so much for your persistence in getting Unpaywall working.

Thanks, Bianca

From: Nicole Kearney notifications@github.com Reply-To: rdmpage/australia reply@reply.github.com Date: Friday, July 12, 2019 at 00:50 To: rdmpage/australia australia@noreply.github.com Cc: "Crowley, Bianca" CrowleyB@si.edu, Mention mention@noreply.github.com Subject: Re: [rdmpage/australia] Get Unpaywall working with BHL (#1)

Hi Bianca and Martin,

To answer your question Bianca, BHL does not currently have PDFs of articles available in a way that makes them discoverable by Unpaywall (or Google Scholar). Yes, users can generate them via the BHL website, and this is great, but we really need to have the PDFs pre-generated and linked to the landing pages in order for them to be discoverable.

Having spoken to Rod about this at length today, I really think this is something we should try to do. It will not only make BHL content discoverable via Unpaywall (which is so critical), it will also make it discoverable via Google Scholar (which I would argue is equally critical).

For example, if you copy this article title into Google Scholarhttps://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=%22Description+of+two+new+Species+of+Didelphis+from+Van+Diemen%27s+Land%22&btnG=, you won’t find the BHL versionhttps://www.biodiversitylibrary.org/part/5582#/summary: "Description of two new Species of Didelphis from Van Diemen's Land" (the first description of the Thylacine, published in 1808). Only the version on Wileyhttps://onlinelibrary.wiley.com/doi/abs/10.1111/j.1096-3642.1818.tb00336.x comes up – and that version is behind a paywall!

So, Unpaywall might have suggested that they might incorporate a work around for the fact that BHL doesn’t have PDFs linked from the landing pages of our articles, but I can’t see Google Scholar doing this. We’re going to have to format our content the way every other publisher does if we want them to be able to find it.

The steps we’d need to undertake are (and here I’m rephrasing what Rod has said below):

To pre-generate PDFs for all segments with DOIs
To create a repository for (a collection of) these PDFs either within BHL’s collection on IA (or have them sit somewhere within BHL)
To add a new field on the segment landing page for the links to these PDFs (this could be displayed as a big shiny “PDF” button). This would be in addition to the existing “View article” link that takes you the article start page in each volume.
To add the “citation_pdf_url” meta tag on the segment landing pages. This is what Unpaywall and Google Scholar are looking for.

Rod has produced code for this for BioStor (it’s on Github) so we can perhaps pick his brains about how we could do this for BHL.

It would be awesome if we could consider doing this…

Cheers, Nicole

This e-mail is solely for the named addressee and may be confidential. You should only read, disclose, transmit, copy, distribute, act in reliance on or commercialise the contents if you are authorised to do so. If you are not the intended recipient of this e-mail, please notify postmaster@museum.vic.gov.au mailto:postmaster@museum.vic.gov.au by email immediately, or notify the sender and then destroy any copy of this message. Views expressed in this email are those of the individual sender, except where specifically stated to be those of an officer of Museums Victoria. Museums Victoria does not represent, warrant or guarantee that the integrity of this communication has been maintained nor that it is free from errors, virus or interference.

From: Roderic Page notifications@github.com Reply-To: rdmpage/australia reply@reply.github.com Date: Friday, 12 July 2019 at 9:54 am To: rdmpage/australia australia@noreply.github.com Cc: Nicole Kearney nkearney@museum.vic.gov.au, Mention mention@noreply.github.com Subject: Re: [rdmpage/australia] Get Unpaywall working with BHL (#1)

@crowleybhttps://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fgithub.com%2fcrowleyb&umid=60361f75-076b-484e-bcf5-2975fa1b7561&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-f8de145df7a7374f256d94eeace47903b7691f2b The issue is that the way BHL does this is SO clunky. A modern journal provides one click access to the PDF from an article page. Unpaywall even provides one click access to the PDF from the article page ON ANOTHER WEBSITE! This is convenient for people who simply want to read the article right away. Manually selecting pages and waiting for a link to be emailed is crazy in this day and age. I think if we were focussed on users rather than process, BHL would:

generate all article PDFs as articles are identified
embed metadata such as DOIs using, e.g. XMP so reference managers can identify the article
Store the PDFs, e.g. on Internet Archive
provide one-click access to these PDFs

I started doing this with BioStor, but didn’t finish as I simply had too much other stuff to do. But it’s straightforward to automate.

The bigger issue here is thinking about users, and trying to make their reading experience as seamless as a that offered by a modern journal publisher. I suspect this will require a bit of a culture shift in BHL, and the current web interface isn’t set up to do this, but I think BHL is making their users’ life harder than it has to be.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fgithub.com%2frdmpage%2faustralia%2fissues%2f1%3femail%5fsource%3dnotifications%26email%5ftoken%3dAHQBYGX5D3OFG74AUGFUL73P67B4DA5CNFSM4HXWCTSKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZYJIII%23issuecomment%2d510694433&umid=60361f75-076b-484e-bcf5-2975fa1b7561&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-1ed0e075e3784b953b50f40e0c070256faaa09d6, or mute the threadhttps://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fgithub.com%2fnotifications%2funsubscribe%2dauth%2fAHQBYGXWS66SWWWFF65HGBTP67B4DANCNFSM4HXWCTSA&umid=60361f75-076b-484e-bcf5-2975fa1b7561&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-8f96721e1e0d2b3e2a1d046d38916d7900d84f05.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Frdmpage%2Faustralia%2Fissues%2F1%3Femail_source%3Dnotifications%26email_token%3DAC47PTPEXZ3U2WIKX6ORUXDP7AESDA5CNFSM4HXWCTSKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZYVISA%23issuecomment-510743624&data=02%7C01%7Ccrowleyb%40si.edu%7C7ab55dd171094e2dc29208d706847e1c%7C989b5e2a14e44efe93b78cdd5fc5d11c%7C0%7C0%7C636985038435820178&sdata=bzF14LUHso0QjIJjeWADGVgWKCpleqzE0johKmgzp9E%3D&reserved=0, or mute the threadhttps://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAC47PTOFURJ23BPBDRD4YWDP7AESDANCNFSM4HXWCTSA&data=02%7C01%7Ccrowleyb%40si.edu%7C7ab55dd171094e2dc29208d706847e1c%7C989b5e2a14e44efe93b78cdd5fc5d11c%7C0%7C0%7C636985038435830173&sdata=6PNLpoKSxNcaui0ZmBh5vhSbGMmYjOnIcSf2Y6RrLoE%3D&reserved=0.

nicolekearney commented 4 years ago

Hi Bianca,

Thank you for following this up. When we wrote this email, it wasn't sounding like Unpaywall would easily be able to create the work-around required to make the open access content on BHL discoverable. Basically the two systems didn't talk to each other. Either BHL needed to change the way we presented our content (following the steps we outlined below) or Unpaywall needed to modify the way they looked for that content, which they did (and I'm immensely grateful that they did this, particularly as it was just for BHL).

So basically, the changes below are no longer required as far as discoverability via Unpaywall is concerned. However, if you see Rod's comments below (highlighted) there are other reasons for presenting BHL journal articles as PDFs.

Kind regards, Nicole

This e-mail is solely for the named addressee and may be confidential. You should only read, disclose, transmit, copy, distribute, act in reliance on or commercialise the contents if you are authorised to do so. If you are not the intended recipient of this e-mail, please notify postmaster@museum.vic.gov.au mailto:postmaster@museum.vic.gov.au by email immediately, or notify the sender and then destroy any copy of this message. Views expressed in this email are those of the individual sender, except where specifically stated to be those of an officer of Museums Victoria. Museums Victoria does not represent, warrant or guarantee that the integrity of this communication has been maintained nor that it is free from errors, virus or interference.

From: Bianca Crowley notifications@github.com Sent: Friday, 30 August 2019 12:27 AM To: rdmpage/australia Cc: Nicole Kearney; Mention Subject: Re: [rdmpage/australia] Get Unpaywall working with BHL (#1)

Hi Nicole, Rod,

I was revisiting your message from below to make sure it didn’t get lost in the mix and have forwarded the issue to BHL’s Gemini system for review by the Tech Team when time allows. For future reference, please email feedback@biodiversitylibrary.orgmailto:feedback@biodiversitylibrary.org to submit requests directly to Gemini.

I’m also not sure if Martin actually got the message below (I thought he mentioned it to me but I cannot see his email below). Either way, Gemini is the best place to send these things. You are also welcome to send them onto me and I’ll get them into Gemini for you. Joel Richard is our new BHL Technical Coordinator and he is going through Gemini with more regularity since he’s taken over the role than folks have done in the past.

Please understand I am doing what I can to get your request to the right place for follow up. Let me know if you have any questions or concerns.

Thanks again so much for your persistence in getting Unpaywall working.

Thanks, Bianca

From: Nicole Kearney notifications@github.com Reply-To: rdmpage/australia reply@reply.github.com Date: Friday, July 12, 2019 at 00:50 To: rdmpage/australia australia@noreply.github.com Cc: "Crowley, Bianca" CrowleyB@si.edu, Mention mention@noreply.github.com Subject: Re: [rdmpage/australia] Get Unpaywall working with BHL (#1)

Hi Bianca and Martin,

To answer your question Bianca, BHL does not currently have PDFs of articles available in a way that makes them discoverable by Unpaywall (or Google Scholar). Yes, users can generate them via the BHL website, and this is great, but we really need to have the PDFs pre-generated and linked to the landing pages in order for them to be discoverable.

Having spoken to Rod about this at length today, I really think this is something we should try to do. It will not only make BHL content discoverable via Unpaywall (which is so critical), it will also make it discoverable via Google Scholar (which I would argue is equally critical).

For example, if you copy this article title into Google Scholar<https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fscholar.google.com%2fscholar%3fhl%3den%26as%5fsdt%3d0%252C5%26q%3d%2522Description%2bof%2btwo%2bnew%2bSpecies%2bof%2bDidelphis%2bfrom%2bVan%2bDiemen%2527s%2bLand%2522%26btnG%3d%3e&umid=ae779d3f-d67b-4945-a8e3-565bcdd52280&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-9854b62357e6b89a7235dcf751227491b880f2f4, you won’t find the BHL version<https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fwww.biodiversitylibrary.org%2fpart%2f5582%23%2fsummary%3e%3a&umid=ae779d3f-d67b-4945-a8e3-565bcdd52280&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-2ba49f41de632c28cbeb9c19cde2cf5460466ad7 "Description of two new Species of Didelphis from Van Diemen's Land" (the first description of the Thylacine, published in 1808). Only the version on Wiley<https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fonlinelibrary.wiley.com%2fdoi%2fabs%2f10.1111%2fj.1096%2d3642.1818.tb00336.x%3e&umid=ae779d3f-d67b-4945-a8e3-565bcdd52280&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-16b799cbffbc1eb21b572c06c3e0b21dc8d68c47 comes up – and that version is behind a paywall!

So, Unpaywall might have suggested that they might incorporate a work around for the fact that BHL doesn’t have PDFs linked from the landing pages of our articles, but I can’t see Google Scholar doing this. We’re going to have to format our content the way every other publisher does if we want them to be able to find it.

The steps we’d need to undertake are (and here I’m rephrasing what Rod has said below):

To pre-generate PDFs for all segments with DOIs
To create a repository for (a collection of) these PDFs either within BHL’s collection on IA (or have them sit somewhere within BHL)
To add a new field on the segment landing page for the links to these PDFs (this could be displayed as a big shiny “PDF” button). This would be in addition to the existing “View article” link that takes you the article start page in each volume.
To add the “citation_pdf_url” meta tag on the segment landing pages. This is what Unpaywall and Google Scholar are looking for.

Rod has produced code for this for BioStor (it’s on Github) so we can perhaps pick his brains about how we could do this for BHL.

It would be awesome if we could consider doing this…

Cheers, Nicole

This e-mail is solely for the named addressee and may be confidential. You should only read, disclose, transmit, copy, distribute, act in reliance on or commercialise the contents if you are authorised to do so. If you are not the intended recipient of this e-mail, please notify postmaster@museum.vic.gov.au mailto:postmaster@museum.vic.gov.au by email immediately, or notify the sender and then destroy any copy of this message. Views expressed in this email are those of the individual sender, except where specifically stated to be those of an officer of Museums Victoria. Museums Victoria does not represent, warrant or guarantee that the integrity of this communication has been maintained nor that it is free from errors, virus or interference.

From: Roderic Page notifications@github.com Reply-To: rdmpage/australia reply@reply.github.com Date: Friday, 12 July 2019 at 9:54 am To: rdmpage/australia australia@noreply.github.com Cc: Nicole Kearney nkearney@museum.vic.gov.au, Mention mention@noreply.github.com Subject: Re: [rdmpage/australia] Get Unpaywall working with BHL (#1)

@crowleybhttps://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fgithub.com%2fcrowleyb&umid=60361f75-076b-484e-bcf5-2975fa1b7561&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-f8de145df7a7374f256d94eeace47903b7691f2b The issue is that the way BHL does this is SO clunky. A modern journal provides one click access to the PDF from an article page. Unpaywall even provides one click access to the PDF from the article page ON ANOTHER WEBSITE! This is convenient for people who simply want to read the article right away. Manually selecting pages and waiting for a link to be emailed is crazy in this day and age. I think if we were focussed on users rather than process, BHL would:

generate all article PDFs as articles are identified
embed metadata such as DOIs using, e.g. XMP so reference managers can identify the article
Store the PDFs, e.g. on Internet Archive
provide one-click access to these PDFs

I started doing this with BioStor, but didn’t finish as I simply had too much other stuff to do. But it’s straightforward to automate.

The bigger issue here is thinking about users, and trying to make their reading experience as seamless as a that offered by a modern journal publisher. I suspect this will require a bit of a culture shift in BHL, and the current web interface isn’t set up to do this, but I think BHL is making their users’ life harder than it has to be.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fgithub.com%2frdmpage%2faustralia%2fissues%2f1%3femail%5fsource%3dnotifications%26email%5ftoken%3dAHQBYGX5D3OFG74AUGFUL73P67B4DA5CNFSM4HXWCTSKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZYJIII%23issuecomment%2d510694433&umid=60361f75-076b-484e-bcf5-2975fa1b7561&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-1ed0e075e3784b953b50f40e0c070256faaa09d6, or mute the threadhttps://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fgithub.com%2fnotifications%2funsubscribe%2dauth%2fAHQBYGXWS66SWWWFF65HGBTP67B4DANCNFSM4HXWCTSA&umid=60361f75-076b-484e-bcf5-2975fa1b7561&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-8f96721e1e0d2b3e2a1d046d38916d7900d84f05.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fnam02.safelinks.protection.outlook.com%2f%3furl%3dhttps%253A%252F%252Fgithub.com%252Frdmpage%252Faustralia%252Fissues%252F1%253Femail%5fsource%253Dnotifications%2526email%5ftoken%253DAC47PTPEXZ3U2WIKX6ORUXDP7AESDA5CNFSM4HXWCTSKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZYVISA%2523issuecomment%2d510743624%26data%3d02%257C01%257Ccrowleyb%2540si.edu%257C7ab55dd171094e2dc29208d706847e1c%257C989b5e2a14e44efe93b78cdd5fc5d11c%257C0%257C0%257C636985038435820178%26sdata%3dbzF14LUHso0QjIJjeWADGVgWKCpleqzE0johKmgzp9E%253D%26reserved%3d0%3e&umid=ae779d3f-d67b-4945-a8e3-565bcdd52280&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-08a87ae735081d87e57d9a586d19dcdad3c71ce8, or mute the thread<https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fnam02.safelinks.protection.outlook.com%2f%3furl%3dhttps%253A%252F%252Fgithub.com%252Fnotifications%252Funsubscribe%2dauth%252FAC47PTOFURJ23BPBDRD4YWDP7AESDANCNFSM4HXWCTSA%26data%3d02%257C01%257Ccrowleyb%2540si.edu%257C7ab55dd171094e2dc29208d706847e1c%257C989b5e2a14e44efe93b78cdd5fc5d11c%257C0%257C0%257C636985038435830173%26sdata%3d6PNLpoKSxNcaui0ZmBh5vhSbGMmYjOnIcSf2Y6RrLoE%253D%26reserved%3d0%3e&umid=ae779d3f-d67b-4945-a8e3-565bcdd52280&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-1fd2dd50ab6df927b952bc3ff3527f3296747c2b.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fgithub.com%2frdmpage%2faustralia%2fissues%2f1%3femail%5fsource%3dnotifications%26email%5ftoken%3dAHQBYGQOQ4NSNFF4OLGP45LQG7MDNA5CNFSM4HXWCTSKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5OVN4A%23issuecomment%2d526210800&umid=ae779d3f-d67b-4945-a8e3-565bcdd52280&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-f953e93a1df09dda7d60d2a546c86f107e0ec39f, or mute the threadhttps://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fgithub.com%2fnotifications%2funsubscribe%2dauth%2fAHQBYGTS2J4FTRD6MRVB4O3QG7MDNANCNFSM4HXWCTSA&umid=ae779d3f-d67b-4945-a8e3-565bcdd52280&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-c1f992ac283f608d51d38e8ef529a951c65caba6.

nicolekearney commented 4 years ago

Hi Nicole, I think Richard is in the process of getting the technical side all worked out. I wanted to weigh in real quick on the logo side. We don't currently put logos of repositories on the site simply because we don't have room...we harvest from over 5000 different ones. We do very greatly value the job you and other repositories are doing, though! We always think about Unpaywall as the easiest link in the chain connecting users to content...the IRs hosting that content are doing the real work, and we try to tell everyone that every chance we get. Keep up the great work! j

On Mon, Aug 26, 2019 at 1:00 PM Richard Orr support@unpaywall.org wrote:

Hi Nicole,

At least in Chrome, we generate requests without any referer header, which might be unusual enough that you could attribute most of any increase in such requests to Unpaywall. Technically this is required behavior https://tools.ietf.org/html/rfc7231#section-5.5.2, but I don't know whether breaking the rules here is a big deal. I'm sorry to say we don't have a lot of bandwidth right now to evaluate it. I'll put this on hold as a feature request. I'm CCing Jason, one of our co-founders, about adding the logo.

Richard Orr Lead Developer, Unpaywall http://unpaywall.org/ OurResearch https://our-research.org/: We build tools to make scholarly research more open, connected, and reusable—for everyone.

https://support.unpaywall.org/public/tickets/130addea2170edd9e4fc198708090fbfc759e801fa0bc184f45c8828c721b636 https://support.unpaywall.org/public/tickets/130addea2170edd9e4fc198708090fbfc759e801fa0bc184f45c8828c721b636

On Thu, 15 Aug at 9:42 PM , Nicole Kearney nkearney@museum.vic.gov.au wrote: Dear Richard,

Thanks again for making it possible for BHL content to be discoverable via your Paywall extension. We’re all still buzzing about it (in fact there will be a blog post published about it on the BHL blog today).

We would very much like to track how much traffic comes to BHL as a result of the Unpaywall fix. However, it seems that BHL is unable to recognise Unpaywall as the source of web traffic; we believe the referrals will be reported as coming from the publisher/etc. sites themselves.

I assume you must have come up against this before. Do you have a way of getting around this? Do you set headers when sending a user from a publisher’s site to an open access article? Can we use these to track the Unpaywall traffic to BHL?

I’d really like to gather these stats and to (hopefully) use them to justify why we need more articles, more article-level metadata and more DOIs to BHL (so that even more of our content can be discoverable via Unpaywall).

Any direction you could give me would be greatly appreciated, Nicole

P.S. I was also wondering whether you might consider adding the BHL to your list of logos next to “Used and trusted by top organizations” on your homepage. J

Nicole Kearney Manager, Biodiversity Heritage Library Australia Digital Life, Museums Victoria PO Box 666, Melbourne VIC 3001 61 3 8341 7779 biodiversitylibrary.org/collection/bhlau https://www.biodiversitylibrary.org/collection/bhlau

From: Richard Orr support@unpaywall.org Sent: Tuesday, 13 August 2019 12:22 PM To: Nicole Kearney nkearney@museum.vic.gov.au Cc: reply@reply.github.com; crowleyb@si.edu Subject: Re: Re: [rdmpage/australia] Get Unpaywall working with BHL (#1)

Hi Nicole,

We've finally fixed the issue and we're now correctly linking BHL pages without PDFs: https://api.unpaywall.org/v2/10.1111/j.1096-3642.1818.tb00336.x?email=richard@impactstory.org https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fapi.unpaywall.org%2fv2%2f10.1111%2fj.1096%2d3642.1818.tb00336.x%3femail%3drichard%40impactstory.org&umid=fb685a6f-6973-4de3-8e2f-b4e796513ab4&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-f812c7d92cb6266b50e866d82beb600c44a207b7. About 43k new pages with public domain or CC license info were linked to DOIs. Thanks again for your patience.

Richard Orr Lead Developer, Unpaywall https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=http%3a%2f%2funpaywall.org&umid=fb685a6f-6973-4de3-8e2f-b4e796513ab4&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-fbda7ea322124c8b286ed4c5c8db39b9abd7ad92 OurResearch https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2four%2dresearch.org&umid=fb685a6f-6973-4de3-8e2f-b4e796513ab4&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-53234f4c968cfbc90c5e493f527a345768d388d6: We build tools to make scholarly research more open, connected, and reusable—for everyone.

https://support.unpaywall.org/public/tickets/130addea2170edd9e4fc198708090fbfc759e801fa0bc184f45c8828c721b636 https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fsupport.unpaywall.org%2fpublic%2ftickets%2f130addea2170edd9e4fc198708090fbfc759e801fa0bc184f45c8828c721b636%c2%a0&umid=fb685a6f-6973-4de3-8e2f-b4e796513ab4&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-9cfb3a179e914a63608228430828c070942f13cb

On Tue, 9 Jul at 4:30 PM , Crowley, Bianca crowleyb@si.edu wrote: Dear Richard,

Thank you for getting back to us and explaining the issue. I can confirm that all of BHL content is available for free and open access and as such the “View Article” link will always lead to the full text of the content. Please add a special case for all BHL pages as you suggest below.

We would very much appreciate hearing from you once this has been completed.

Thank you again for taking the time to review our content, eliminate the duplicate entry, and granting us an exception to the rule. Please let me know if you run into any further issues with BHL content and I will do my best to assist.

Regards, Bianca

Bianca Crowley Digital Collections Manager Digital Programs & Initiatives Division crowleyb@si.edu | 202.633.2239

[image: Image removed by sender. EmailSignature_option1_noTag_RGB]

From: Richard Orr support@unpaywall.org Reply-To: Richard Orr support@unpaywall.org Date: Tuesday, July 9, 2019 at 17:16 To: Nicole Kearney nkearney@museum.vic.gov.au Cc: "reply@reply.github.com" reply@reply.github.com, "Crowley, Bianca" CrowleyB@si.edu Subject: Re: Re: [rdmpage/australia] Get Unpaywall working with BHL (#1)

Hi Nicole,

Thank you for your patience and I'm sorry for not replying sooner. I've been waiting until the problem is solved, which it isn't yet, but I do want to at least let you know what the problem is and that we haven't forgotten about it.

Out problem with the example you provided, and others like https://www.biodiversitylibrary.org/part/281352, https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fnam02.safelinks.protection.outlook.com%2f%3furl%3dhttps%253A%252F%252Fwww.biodiversitylibrary.org%252Fpart%252F281352%252C%26data%3d02%257C01%257Ccrowleyb%2540si.edu%257C9bb8b83952f24a34243e08d704b2b88d%257C989b5e2a14e44efe93b78cdd5fc5d11c%257C0%257C0%257C636983037995244991%26sdata%3d%252BG6%252FyzNjbrlAyvxfMPEOLkTb7nOkVv0us0Kd6vBH3bM%253D%26reserved%3d0&umid=fb685a6f-6973-4de3-8e2f-b4e796513ab4&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-85c859e3d1fc3997735962782bafbeb8bfa649b2 is that on repository pages we look for a PDF link to confirm that the document is actually available. We're not able to recognize the embedded reader on those pages, so they don't get used.

Does the "View article" link always lead to a full copy? If so, I think we can just add a special case for these pages.

The duplicate endpoint doesn't create a problem aside from some wasted harvesting effort on our end, but I'll go ahead and remove one. https://unpaywall.org/sources/repository/q6caunfjwsunh6wiqwpg https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fnam02.safelinks.protection.outlook.com%2f%3furl%3dhttps%253A%252F%252Funpaywall.org%252Fsources%252Frepository%252Fq6caunfjwsunh6wiqwpg%26data%3d02%257C01%257Ccrowleyb%2540si.edu%257C9bb8b83952f24a34243e08d704b2b88d%257C989b5e2a14e44efe93b78cdd5fc5d11c%257C0%257C1%257C636983037995244991%26sdata%3dMSS3749mAbL07IW9zHqfbxGQnWUsvPnpXoiMzRN9pmM%253D%26reserved%3d0&umid=fb685a6f-6973-4de3-8e2f-b4e796513ab4&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-033eca5d498bb716bcbd9608218a208577f75440 will be the one we keep.

Thanks,

Richard Orr Lead Developer, Unpaywall Impactstory https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fnam02.safelinks.protection.outlook.com%2f%3furl%3dhttp%253A%252F%252Fimpactstory.org%252F%26data%3d02%257C01%257Ccrowleyb%2540si.edu%257C9bb8b83952f24a34243e08d704b2b88d%257C989b5e2a14e44efe93b78cdd5fc5d11c%257C0%257C1%257C636983037995254982%26sdata%3diWaG5uL1AtKnIEqc4LQEIkLzQxDHtz7D8Mvir8qIiRA%253D%26reserved%3d0&umid=fb685a6f-6973-4de3-8e2f-b4e796513ab4&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-5fc612e6098377485469c76a167fa19fc112ed2d: We make tools to power the Open Science revolution

https://support.unpaywall.org/public/tickets/130addea2170edd9e4fc198708090fbfc759e801fa0bc184f45c8828c721b636 https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fnam02.safelinks.protection.outlook.com%2f%3furl%3dhttps%253A%252F%252Fsupport.unpaywall.org%252Fpublic%252Ftickets%252F130addea2170edd9e4fc198708090fbfc759e801fa0bc184f45c8828c721b636%26data%3d02%257C01%257Ccrowleyb%2540si.edu%257C9bb8b83952f24a34243e08d704b2b88d%257C989b5e2a14e44efe93b78cdd5fc5d11c%257C0%257C1%257C636983037995254982%26sdata%3dqbJLBuBxFro8yCl4pzukWJUrJ3J2%252F6LBMhgO8YCYP1s%253D%26reserved%3d0&umid=fb685a6f-6973-4de3-8e2f-b4e796513ab4&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-6a4ef169840846748f208bee4d0dc42b4a1d7093

This e-mail is solely for the named addressee and may be confidential. You should only read, disclose, transmit, copy, distribute, act in reliance on or commercialise the contents if you are authorised to do so. If you are not the intended recipient of this e-mail, please notify postmaster@museum.vic.gov.au postmaster@museum.vic.gov.au%20 by email immediately, or notify the sender and then destroy any copy of this message. Views expressed in this email are those of the individual sender, except where specifically stated to be those of an officer of Museums Victoria. Museums Victoria does not represent, warrant or guarantee that the integrity of this communication has been maintained nor that it is free from errors, virus or interference.

443:1048800

-- Jason Priem, cofounder Our Research https://our-research.org/: We build tools to make scholarly research more open, connected, and reusable—for everyone. follow at @jasonpriem https://twitter.com/jasonpriem, @our_research https://twitter.com/our_research, and @unpaywall https://twitter.com/unpaywall

nicolekearney commented 4 years ago

That’s completely understandable. I suppose 5000 logos would look a bit messy on your homepage – and would be a logistical nightmare! Thanks again for all you’ve done to make BHL’s content is now discoverable via Unpaywall.

Nicole Kearney Manager, Biodiversity Heritage Library Australia Digital Life, Museums Victoria PO Box 666, Melbourne VIC 3001 61 3 8341 7779 biodiversitylibrary.org/collection/bhlauhttps://www.biodiversitylibrary.org/collection/bhlau

From: Jason Priem jason@ourresearch.org Sent: Sunday, 1 September 2019 2:47 AM To: Richard Orr support@unpaywall.org Cc: Nicole Kearney nkearney@museum.vic.gov.au; reply@reply.github.com; crowleyb@si.edu; costantinog@si.edu Subject: Re: Re: [rdmpage/australia] Get Unpaywall working with BHL (#1)

Hi Nicole, I think Richard is in the process of getting the technical side all worked out. I wanted to weigh in real quick on the logo side. We don't currently put logos of repositories on the site simply because we don't have room...we harvest from over 5000 different ones. We do very greatly value the job you and other repositories are doing, though! We always think about Unpaywall as the easiest link in the chain connecting users to content...the IRs hosting that content are doing the real work, and we try to tell everyone that every chance we get. Keep up the great work! j

On Mon, Aug 26, 2019 at 1:00 PM Richard Orr support@unpaywall.org<mailto:support@unpaywall.org> wrote: Hi Nicole,

At least in Chrome, we generate requests without any referer header, which might be unusual enough that you could attribute most of any increase in such requests to Unpaywall. Technically this is required behaviorhttps://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2ftools.ietf.org%2fhtml%2frfc7231%23section%2d5.5.2&umid=2b68d43e-cb47-4ada-899a-487335aa0cf2&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-b13d4ce90ac823decc9bf614afb1af3cbd6e2cb8, but I don't know whether breaking the rules here is a big deal. I'm sorry to say we don't have a lot of bandwidth right now to evaluate it. I'll put this on hold as a feature request. I'm CCing Jason, one of our co-founders, about adding the logo.

Richard Orr Lead Developer, Unpaywallhttps://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=http%3a%2f%2funpaywall.org&umid=2b68d43e-cb47-4ada-899a-487335aa0cf2&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-82b613478a1e7d69f4c73584fd40074b170b6415 OurResearchhttps://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2four%2dresearch.org&umid=2b68d43e-cb47-4ada-899a-487335aa0cf2&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-66cc0209657b590424f69eaeb7274b216044482f: We build tools to make scholarly research more open, connected, and reusable—for everyone.

https://support.unpaywall.org/public/tickets/130addea2170edd9e4fc198708090fbfc759e801fa0bc184f45c8828c721b636 https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fsupport.unpaywall.org%2fpublic%2ftickets%2f130addea2170edd9e4fc198708090fbfc759e801fa0bc184f45c8828c721b636&umid=2b68d43e-cb47-4ada-899a-487335aa0cf2&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-b4c9c14ea445c7dce013d679262081a25a9109b4 On Thu, 15 Aug at 9:42 PM , Nicole Kearney nkearney@museum.vic.gov.au<mailto:nkearney@museum.vic.gov.au> wrote: Dear Richard,

Thanks again for making it possible for BHL content to be discoverable via your Paywall extension. We’re all still buzzing about it (in fact there will be a blog post published about it on the BHL blog today).

We would very much like to track how much traffic comes to BHL as a result of the Unpaywall fix. However, it seems that BHL is unable to recognise Unpaywall as the source of web traffic; we believe the referrals will be reported as coming from the publisher/etc. sites themselves.

I assume you must have come up against this before. Do you have a way of getting around this? Do you set headers when sending a user from a publisher’s site to an open access article? Can we use these to track the Unpaywall traffic to BHL?

I’d really like to gather these stats and to (hopefully) use them to justify why we need more articles, more article-level metadata and more DOIs to BHL (so that even more of our content can be discoverable via Unpaywall).

Any direction you could give me would be greatly appreciated, Nicole

P.S. I was also wondering whether you might consider adding the BHL to your list of logos next to “Used and trusted by top organizations” on your homepage. ☺

Nicole Kearney Manager, Biodiversity Heritage Library Australia Digital Life, Museums Victoria PO Box 666, Melbourne VIC 3001 61 3 8341 7779 biodiversitylibrary.org/collection/bhlauhttps://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fwww.biodiversitylibrary.org%2fcollection%2fbhlau&umid=2b68d43e-cb47-4ada-899a-487335aa0cf2&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-0fe2b925cba216a2110990d8aefbb33688e11ce0

From: Richard Orr support@unpaywall.org<mailto:support@unpaywall.org> Sent: Tuesday, 13 August 2019 12:22 PM To: Nicole Kearney nkearney@museum.vic.gov.au<mailto:nkearney@museum.vic.gov.au> Cc: reply@reply.github.commailto:reply%2Bahqbygquuayao5kj5s2kref3d6mrxevbnhhbwjysie@reply.github.com; crowleyb@si.edumailto:crowleyb@si.edu Subject: Re: Re: [rdmpage/australia] Get Unpaywall working with BHL (#1)

Hi Nicole,

We've finally fixed the issue and we're now correctly linking BHL pages without PDFs: https://api.unpaywall.org/v2/10.1111/j.1096-3642.1818.tb00336.x?email=richard@impactstory.org https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fapi.unpaywall.org%2fv2%2f10.1111%2fj.1096%2d3642.1818.tb00336.x%3femail%3drichard%40impactstory.org&umid=2b68d43e-cb47-4ada-899a-487335aa0cf2&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-8f81d20376d5f45f2dfc9d53789f24f03d2d7c3c. About 43k new pages with public domain or CC license info were linked to DOIs. Thanks again for your patience.

Richard Orr Lead Developer, Unpaywallhttps://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=http%3a%2f%2funpaywall.org&umid=2b68d43e-cb47-4ada-899a-487335aa0cf2&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-82b613478a1e7d69f4c73584fd40074b170b6415 OurResearchhttps://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2four%2dresearch.org&umid=2b68d43e-cb47-4ada-899a-487335aa0cf2&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-66cc0209657b590424f69eaeb7274b216044482f: We build tools to make scholarly research more open, connected, and reusable—for everyone.

https://support.unpaywall.org/public/tickets/130addea2170edd9e4fc198708090fbfc759e801fa0bc184f45c8828c721b636 https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fsupport.unpaywall.org%2fpublic%2ftickets%2f130addea2170edd9e4fc198708090fbfc759e801fa0bc184f45c8828c721b636%c2%a0&umid=2b68d43e-cb47-4ada-899a-487335aa0cf2&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-a830ed4f5c1992e6c1532daac82dedeac01a12a1 On Tue, 9 Jul at 4:30 PM , Crowley, Bianca crowleyb@si.edu<mailto:crowleyb@si.edu> wrote: Dear Richard,

Thank you for getting back to us and explaining the issue. I can confirm that all of BHL content is available for free and open access and as such the “View Article” link will always lead to the full text of the content. Please add a special case for all BHL pages as you suggest below.

We would very much appreciate hearing from you once this has been completed.

Thank you again for taking the time to review our content, eliminate the duplicate entry, and granting us an exception to the rule. Please let me know if you run into any further issues with BHL content and I will do my best to assist.

Regards, Bianca

Bianca Crowley Digital Collections Manager Digital Programs & Initiatives Division crowleyb@si.edumailto:crowleyb@si.edu | 202.633.2239

[Image removed by sender. Image removed by sender. EmailSignature_option1_noTag_RGB]

From: Richard Orr support@unpaywall.org<mailto:support@unpaywall.org> Reply-To: Richard Orr support@unpaywall.org<mailto:support@unpaywall.org> Date: Tuesday, July 9, 2019 at 17:16 To: Nicole Kearney nkearney@museum.vic.gov.au<mailto:nkearney@museum.vic.gov.au> Cc: "reply@reply.github.commailto:reply%2Bahqbygquuayao5kj5s2kref3d6mrxevbnhhbwjysie@reply.github.com" reply@reply.github.com<mailto:reply%2Bahqbygquuayao5kj5s2kref3d6mrxevbnhhbwjysie@reply.github.com>, "Crowley, Bianca" CrowleyB@si.edu<mailto:CrowleyB@si.edu> Subject: Re: Re: [rdmpage/australia] Get Unpaywall working with BHL (#1)

Hi Nicole,

Thank you for your patience and I'm sorry for not replying sooner. I've been waiting until the problem is solved, which it isn't yet, but I do want to at least let you know what the problem is and that we haven't forgotten about it.

Out problem with the example you provided, and others like https://www.biodiversitylibrary.org/part/281352,https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fnam02.safelinks.protection.outlook.com%2f%3furl%3dhttps%253A%252F%252Fwww.biodiversitylibrary.org%252Fpart%252F281352%252C%26data%3d02%257C01%257Ccrowleyb%2540si.edu%257C9bb8b83952f24a34243e08d704b2b88d%257C989b5e2a14e44efe93b78cdd5fc5d11c%257C0%257C0%257C636983037995244991%26sdata%3d%252BG6%252FyzNjbrlAyvxfMPEOLkTb7nOkVv0us0Kd6vBH3bM%253D%26reserved%3d0&umid=2b68d43e-cb47-4ada-899a-487335aa0cf2&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-902cf4b4545352ab628d199705d277bc2c5c51eb is that on repository pages we look for a PDF link to confirm that the document is actually available. We're not able to recognize the embedded reader on those pages, so they don't get used.

Does the "View article" link always lead to a full copy? If so, I think we can just add a special case for these pages.

The duplicate endpoint doesn't create a problem aside from some wasted harvesting effort on our end, but I'll go ahead and remove one. https://unpaywall.org/sources/repository/q6caunfjwsunh6wiqwpg https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fnam02.safelinks.protection.outlook.com%2f%3furl%3dhttps%253A%252F%252Funpaywall.org%252Fsources%252Frepository%252Fq6caunfjwsunh6wiqwpg%26data%3d02%257C01%257Ccrowleyb%2540si.edu%257C9bb8b83952f24a34243e08d704b2b88d%257C989b5e2a14e44efe93b78cdd5fc5d11c%257C0%257C1%257C636983037995244991%26sdata%3dMSS3749mAbL07IW9zHqfbxGQnWUsvPnpXoiMzRN9pmM%253D%26reserved%3d0&umid=2b68d43e-cb47-4ada-899a-487335aa0cf2&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-f3bf644554f38236f3a6f430846b81e8d223ac3a will be the one we keep.

Thanks,

Richard Orr Lead Developer, Unpaywall Impactstoryhttps://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fnam02.safelinks.protection.outlook.com%2f%3furl%3dhttp%253A%252F%252Fimpactstory.org%252F%26data%3d02%257C01%257Ccrowleyb%2540si.edu%257C9bb8b83952f24a34243e08d704b2b88d%257C989b5e2a14e44efe93b78cdd5fc5d11c%257C0%257C1%257C636983037995254982%26sdata%3diWaG5uL1AtKnIEqc4LQEIkLzQxDHtz7D8Mvir8qIiRA%253D%26reserved%3d0&umid=2b68d43e-cb47-4ada-899a-487335aa0cf2&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-5419d8f3168f686ebcecde0b08ebc28863ede2f4: We make tools to power the Open Science revolution

https://support.unpaywall.org/public/tickets/130addea2170edd9e4fc198708090fbfc759e801fa0bc184f45c8828c721b636 https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fnam02.safelinks.protection.outlook.com%2f%3furl%3dhttps%253A%252F%252Fsupport.unpaywall.org%252Fpublic%252Ftickets%252F130addea2170edd9e4fc198708090fbfc759e801fa0bc184f45c8828c721b636%26data%3d02%257C01%257Ccrowleyb%2540si.edu%257C9bb8b83952f24a34243e08d704b2b88d%257C989b5e2a14e44efe93b78cdd5fc5d11c%257C0%257C1%257C636983037995254982%26sdata%3dqbJLBuBxFro8yCl4pzukWJUrJ3J2%252F6LBMhgO8YCYP1s%253D%26reserved%3d0&umid=2b68d43e-cb47-4ada-899a-487335aa0cf2&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-3cff483ed5eac3fd18a972f91901a008b6833c73

This e-mail is solely for the named addressee and may be confidential. You should only read, disclose, transmit, copy, distribute, act in reliance on or commercialise the contents if you are authorised to do so. If you are not the intended recipient of this e-mail, please notify postmaster@museum.vic.gov.au mailto:postmaster@museum.vic.gov.au%20 by email immediately, or notify the sender and then destroy any copy of this message. Views expressed in this email are those of the individual sender, except where specifically stated to be those of an officer of Museums Victoria. Museums Victoria does not represent, warrant or guarantee that the integrity of this communication has been maintained nor that it is free from errors, virus or interference.

443:1048800

-- Jason Priem, cofounder Our Researchhttps://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2four%2dresearch.org&umid=2b68d43e-cb47-4ada-899a-487335aa0cf2&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-66cc0209657b590424f69eaeb7274b216044482f: We build tools to make scholarly research more open, connected, and reusable—for everyone. follow at @jasonpriemhttps://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2ftwitter.com%2fjasonpriem&umid=2b68d43e-cb47-4ada-899a-487335aa0cf2&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-af2a6ed4a21add9485f670c6d7321b44a0608fd7, @our_researchhttps://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2ftwitter.com%2four%5fresearch&umid=2b68d43e-cb47-4ada-899a-487335aa0cf2&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-720565f9c1d25293b9db2992635fe833730a04de, and @unpaywallhttps://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2ftwitter.com%2funpaywall&umid=2b68d43e-cb47-4ada-899a-487335aa0cf2&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-3eafc432b8430597037477a575c919d79f6bc4f7

This e-mail is solely for the named addressee and may be confidential. You should only read, disclose, transmit, copy, distribute, act in reliance on or commercialise the contents if you are authorised to do so. If you are not the intended recipient of this e-mail, please notify postmaster@museum.vic.gov.au mailto:postmaster@museum.vic.gov.au by email immediately, or notify the sender and then destroy any copy of this message. Views expressed in this email are those of the individual sender, except where specifically stated to be those of an officer of Museums Victoria. Museums Victoria does not represent, warrant or guarantee that the integrity of this communication has been maintained nor that it is free from errors, virus or interference.

nicolekearney commented 4 years ago

Glad we could be of service! Thanks for all the great work y'all do too! j

On Sun, Sep 1, 2019 at 6:32 PM Nicole Kearney nkearney@museum.vic.gov.au wrote:

That’s completely understandable. I suppose 5000 logos would look a bit messy on your homepage – and would be a logistical nightmare! Thanks again for all you’ve done to make BHL’s content is now discoverable via Unpaywall.

Nicole Kearney

Manager, Biodiversity Heritage Library Australia

Digital Life, Museums Victoria PO Box 666, Melbourne VIC 3001

61 3 8341 7779

biodiversitylibrary.org/collection/bhlau https://www.biodiversitylibrary.org/collection/bhlau

From: Jason Priem jason@ourresearch.org Sent: Sunday, 1 September 2019 2:47 AM To: Richard Orr support@unpaywall.org Cc: Nicole Kearney nkearney@museum.vic.gov.au; reply@reply.github.com; crowleyb@si.edu; costantinog@si.edu Subject: Re: Re: [rdmpage/australia] Get Unpaywall working with BHL (#1)

Hi Nicole,

I think Richard is in the process of getting the technical side all worked out. I wanted to weigh in real quick on the logo side. We don't currently put logos of repositories on the site simply because we don't have room...we harvest from over 5000 different ones. We do very greatly value the job you and other repositories are doing, though! We always think about Unpaywall as the easiest link in the chain connecting users to content...the IRs hosting that content are doing the real work, and we try to tell everyone that every chance we get. Keep up the great work!

j

On Mon, Aug 26, 2019 at 1:00 PM Richard Orr support@unpaywall.org wrote:

Hi Nicole,

At least in Chrome, we generate requests without any referer header, which might be unusual enough that you could attribute most of any increase in such requests to Unpaywall. Technically this is required behavior https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2ftools.ietf.org%2fhtml%2frfc7231%23section%2d5.5.2&umid=2b68d43e-cb47-4ada-899a-487335aa0cf2&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-b13d4ce90ac823decc9bf614afb1af3cbd6e2cb8, but I don't know whether breaking the rules here is a big deal. I'm sorry to say we don't have a lot of bandwidth right now to evaluate it. I'll put this on hold as a feature request.

I'm CCing Jason, one of our co-founders, about adding the logo.

Richard Orr

Lead Developer, Unpaywall https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=http%3a%2f%2funpaywall.org&umid=2b68d43e-cb47-4ada-899a-487335aa0cf2&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-82b613478a1e7d69f4c73584fd40074b170b6415

OurResearch https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2four%2dresearch.org&umid=2b68d43e-cb47-4ada-899a-487335aa0cf2&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-66cc0209657b590424f69eaeb7274b216044482f: We build tools to make scholarly research more open, connected,

and reusable—for everyone.

https://support.unpaywall.org/public/tickets/130addea2170edd9e4fc198708090fbfc759e801fa0bc184f45c8828c721b636 https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fsupport.unpaywall.org%2fpublic%2ftickets%2f130addea2170edd9e4fc198708090fbfc759e801fa0bc184f45c8828c721b636&umid=2b68d43e-cb47-4ada-899a-487335aa0cf2&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-b4c9c14ea445c7dce013d679262081a25a9109b4

On Thu, 15 Aug at 9:42 PM , Nicole Kearney nkearney@museum.vic.gov.au wrote:

Dear Richard,

Thanks again for making it possible for BHL content to be discoverable via your Paywall extension. We’re all still buzzing about it (in fact there will be a blog post published about it on the BHL blog today).

We would very much like to track how much traffic comes to BHL as a result of the Unpaywall fix. However, it seems that BHL is unable to recognise Unpaywall as the source of web traffic; we believe the referrals will be reported as coming from the publisher/etc. sites themselves.

I assume you must have come up against this before. Do you have a way of getting around this? Do you set headers when sending a user from a publisher’s site to an open access article? Can we use these to track the Unpaywall traffic to BHL?

I’d really like to gather these stats and to (hopefully) use them to justify why we need more articles, more article-level metadata and more DOIs to BHL (so that even more of our content can be discoverable via Unpaywall).

Any direction you could give me would be greatly appreciated, Nicole

P.S. I was also wondering whether you might consider adding the BHL to your list of logos next to “Used and trusted by top organizations” on your homepage. J

Nicole Kearney

Manager, Biodiversity Heritage Library Australia

Digital Life, Museums Victoria PO Box 666, Melbourne VIC 3001

61 3 8341 7779

biodiversitylibrary.org/collection/bhlau https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fwww.biodiversitylibrary.org%2fcollection%2fbhlau&umid=2b68d43e-cb47-4ada-899a-487335aa0cf2&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-0fe2b925cba216a2110990d8aefbb33688e11ce0

From: Richard Orr support@unpaywall.org Sent: Tuesday, 13 August 2019 12:22 PM To: Nicole Kearney nkearney@museum.vic.gov.au Cc: reply@reply.github.com; crowleyb@si.edu Subject: Re: Re: [rdmpage/australia] Get Unpaywall working with BHL (#1)

Hi Nicole,

We've finally fixed the issue and we're now correctly linking BHL pages without PDFs: https://api.unpaywall.org/v2/10.1111/j.1096-3642.1818.tb00336.x?email=richard@impactstory.org https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fapi.unpaywall.org%2fv2%2f10.1111%2fj.1096%2d3642.1818.tb00336.x%3femail%3drichard%40impactstory.org&umid=2b68d43e-cb47-4ada-899a-487335aa0cf2&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-8f81d20376d5f45f2dfc9d53789f24f03d2d7c3c. About 43k new pages with public domain or CC license info were linked to DOIs. Thanks again for your patience.

Richard Orr

Lead Developer, Unpaywall https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=http%3a%2f%2funpaywall.org&umid=2b68d43e-cb47-4ada-899a-487335aa0cf2&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-82b613478a1e7d69f4c73584fd40074b170b6415

OurResearch https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2four%2dresearch.org&umid=2b68d43e-cb47-4ada-899a-487335aa0cf2&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-66cc0209657b590424f69eaeb7274b216044482f: We build tools to make scholarly research more open, connected,

and reusable—for everyone.

https://support.unpaywall.org/public/tickets/130addea2170edd9e4fc198708090fbfc759e801fa0bc184f45c8828c721b636 https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fsupport.unpaywall.org%2fpublic%2ftickets%2f130addea2170edd9e4fc198708090fbfc759e801fa0bc184f45c8828c721b636%c2%a0&umid=2b68d43e-cb47-4ada-899a-487335aa0cf2&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-a830ed4f5c1992e6c1532daac82dedeac01a12a1

On Tue, 9 Jul at 4:30 PM , Crowley, Bianca crowleyb@si.edu wrote:

Dear Richard,

Thank you for getting back to us and explaining the issue. I can confirm that all of BHL content is available for free and open access and as such the “View Article” link will always lead to the full text of the content. Please add a special case for all BHL pages as you suggest below.

We would very much appreciate hearing from you once this has been completed.

Thank you again for taking the time to review our content, eliminate the duplicate entry, and granting us an exception to the rule. Please let me know if you run into any further issues with BHL content and I will do my best to assist.

Regards,

Bianca

Bianca Crowley

Digital Collections Manager

Digital Programs & Initiatives Division

crowleyb@si.edu | 202.633.2239

[image: Image removed by sender. Image removed by sender. EmailSignature_option1_noTag_RGB]

From: Richard Orr support@unpaywall.org Reply-To: Richard Orr support@unpaywall.org Date: Tuesday, July 9, 2019 at 17:16 To: Nicole Kearney nkearney@museum.vic.gov.au Cc: "reply@reply.github.com" reply@reply.github.com, "Crowley, Bianca" CrowleyB@si.edu Subject: Re: Re: [rdmpage/australia] Get Unpaywall working with BHL (#1)

Hi Nicole,

Thank you for your patience and I'm sorry for not replying sooner. I've been waiting until the problem is solved, which it isn't yet, but I do want to at least let you know what the problem is and that we haven't forgotten about it.

Out problem with the example you provided, and others like https://www.biodiversitylibrary.org/part/281352, https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fnam02.safelinks.protection.outlook.com%2f%3furl%3dhttps%253A%252F%252Fwww.biodiversitylibrary.org%252Fpart%252F281352%252C%26data%3d02%257C01%257Ccrowleyb%2540si.edu%257C9bb8b83952f24a34243e08d704b2b88d%257C989b5e2a14e44efe93b78cdd5fc5d11c%257C0%257C0%257C636983037995244991%26sdata%3d%252BG6%252FyzNjbrlAyvxfMPEOLkTb7nOkVv0us0Kd6vBH3bM%253D%26reserved%3d0&umid=2b68d43e-cb47-4ada-899a-487335aa0cf2&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-902cf4b4545352ab628d199705d277bc2c5c51eb is that on repository pages we look for a PDF link to confirm that the document is actually available. We're not able to recognize the embedded reader on those pages, so they don't get used.

Does the "View article" link always lead to a full copy? If so, I think we can just add a special case for these pages.

The duplicate endpoint doesn't create a problem aside from some wasted harvesting effort on our end, but I'll go ahead and remove one. https://unpaywall.org/sources/repository/q6caunfjwsunh6wiqwpg https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fnam02.safelinks.protection.outlook.com%2f%3furl%3dhttps%253A%252F%252Funpaywall.org%252Fsources%252Frepository%252Fq6caunfjwsunh6wiqwpg%26data%3d02%257C01%257Ccrowleyb%2540si.edu%257C9bb8b83952f24a34243e08d704b2b88d%257C989b5e2a14e44efe93b78cdd5fc5d11c%257C0%257C1%257C636983037995244991%26sdata%3dMSS3749mAbL07IW9zHqfbxGQnWUsvPnpXoiMzRN9pmM%253D%26reserved%3d0&umid=2b68d43e-cb47-4ada-899a-487335aa0cf2&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-f3bf644554f38236f3a6f430846b81e8d223ac3a will be the one we keep.

Thanks,

Richard Orr

Lead Developer, Unpaywall

Impactstory https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fnam02.safelinks.protection.outlook.com%2f%3furl%3dhttp%253A%252F%252Fimpactstory.org%252F%26data%3d02%257C01%257Ccrowleyb%2540si.edu%257C9bb8b83952f24a34243e08d704b2b88d%257C989b5e2a14e44efe93b78cdd5fc5d11c%257C0%257C1%257C636983037995254982%26sdata%3diWaG5uL1AtKnIEqc4LQEIkLzQxDHtz7D8Mvir8qIiRA%253D%26reserved%3d0&umid=2b68d43e-cb47-4ada-899a-487335aa0cf2&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-5419d8f3168f686ebcecde0b08ebc28863ede2f4: We make tools to power the Open Science revolution

https://support.unpaywall.org/public/tickets/130addea2170edd9e4fc198708090fbfc759e801fa0bc184f45c8828c721b636 https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fnam02.safelinks.protection.outlook.com%2f%3furl%3dhttps%253A%252F%252Fsupport.unpaywall.org%252Fpublic%252Ftickets%252F130addea2170edd9e4fc198708090fbfc759e801fa0bc184f45c8828c721b636%26data%3d02%257C01%257Ccrowleyb%2540si.edu%257C9bb8b83952f24a34243e08d704b2b88d%257C989b5e2a14e44efe93b78cdd5fc5d11c%257C0%257C1%257C636983037995254982%26sdata%3dqbJLBuBxFro8yCl4pzukWJUrJ3J2%252F6LBMhgO8YCYP1s%253D%26reserved%3d0&umid=2b68d43e-cb47-4ada-899a-487335aa0cf2&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-3cff483ed5eac3fd18a972f91901a008b6833c73

This e-mail is solely for the named addressee and may be confidential. You should only read, disclose, transmit, copy, distribute, act in reliance on or commercialise the contents if you are authorised to do so. If you are not the intended recipient of this e-mail, please notify postmaster@museum.vic.gov.au postmaster@museum.vic.gov.au%20 by email immediately, or notify the sender and then destroy any copy of this message. Views expressed in this email are those of the individual sender, except where specifically stated to be those of an officer of Museums Victoria. Museums Victoria does not represent, warrant or guarantee that the integrity of this communication has been maintained nor that it is free from errors, virus or interference.

443:1048800

--

Jason Priem, cofounder

Our Research https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2four%2dresearch.org&umid=2b68d43e-cb47-4ada-899a-487335aa0cf2&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-66cc0209657b590424f69eaeb7274b216044482f: We build tools to make scholarly research more open, connected, and reusable—for everyone.

follow at @jasonpriem https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2ftwitter.com%2fjasonpriem&umid=2b68d43e-cb47-4ada-899a-487335aa0cf2&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-af2a6ed4a21add9485f670c6d7321b44a0608fd7 , @our_research https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2ftwitter.com%2four%5fresearch&umid=2b68d43e-cb47-4ada-899a-487335aa0cf2&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-720565f9c1d25293b9db2992635fe833730a04de, and @unpaywall https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2ftwitter.com%2funpaywall&umid=2b68d43e-cb47-4ada-899a-487335aa0cf2&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-3eafc432b8430597037477a575c919d79f6bc4f7

This e-mail is solely for the named addressee and may be confidential. You should only read, disclose, transmit, copy, distribute, act in reliance on or commercialise the contents if you are authorised to do so. If you are not the intended recipient of this e-mail, please notify postmaster@museum.vic.gov.au by email immediately, or notify the sender and then destroy any copy of this message. Views expressed in this email are those of the individual sender, except where specifically stated to be those of an officer of Museums Victoria. Museums Victoria does not represent, warrant or guarantee that the integrity of this communication has been maintained nor that it is free from errors, virus or interference.

-- Jason Priem, cofounder Our Research https://our-research.org/: We build tools to make scholarly research more open, connected, and reusable—for everyone. follow at @jasonpriem https://twitter.com/jasonpriem, @our_research https://twitter.com/our_research, and @unpaywall https://twitter.com/unpaywall

rdmpage / australia

Get Unpaywall working with BHL #1

1 We'll need to see what happened with the first batch that was registered and why it changed.

2 it will also be useful now to make sure there is only one instance of BHL represented in Unpaywall

1https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fgithub.com%2frdmpage%2faustralia%2fissues%2f1&umid=db449783-476b-4de4-a58b-637002129e4e&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-c9cef49640d5dc2b77d0c2ca2082573dd1e891f5

2https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fgithub.com%2frdmpage%2faustralia%2fissues%2f2&umid=db449783-476b-4de4-a58b-637002129e4e&auth=89a422ce48cf9afc268cabe806cc53ea452e36bd-9036f11bfa8b91c9b8aca6725f62a785d8e89634