sedimentation-fault
commented
4 years ago I want to commend you and thank you for putting so much thought into a naming system of papers and their related information, like bibtexs. This is by no means self-evident - and in fact I have seen it only at ACL. It is incredible, but you are the only people on the planet who name related resources with the same basename! This puts you light years ahead of your time!
Let me explain:
Suppose I look at the aclweb.org site and say to myself: "WOW! What an incredible treasure. I would love to download it all and have it in my local paper library for my reading pleasure!". Well, that's easy. I remember looking at it 10 years ago - or even further back in the past. It has always been easy to "get them all". But that's only part of the story. Having PDFs named like "pennington-etal-2014-glove.pdf" does not help at all - you must rename them to some naming scheme amenable to searching, e.g.:
Venue Volume Issue Year DOI Authors Title
For example, for the above paper:
Proceedings 2014 Conference on Empirical Methods in Natural Language Processing EMNLP 2014 [doi 10.3115%2Fv1%2FD14-1162] Pennington, Jeffrey; Socher, Richard; Manning, Christopher -- Glove - Global Vectors for Word Representation.pdf
Notice that you can reconstruct a basic bibtex from the above name, knowing that semicolons delimit author names, the tile comes after ' -- ', the DOI is the URL-encoded string XXX in '[doi XXX]', the year is the 4-digit string before the DOI part and "Venue" is before that.
For this to work, you need bibliographic information for each paper, say in the form of a .bib file. You have that - everybody has that. But what you have - and everybody else is still missing - is this:
The paper and its associated bibliographic information have the same basename! That is, if the above paper has a URL
https://www.aclweb.org/anthology/D14-1162
then I know that the paper is at
https://www.aclweb.org/anthology/D14-1162.pdf
and its associated bibtex at
https://www.aclweb.org/anthology/D14-1162.bib
I can get those two just by looking at the 'url={...}' lines of the 'cumulative' bibtex at
https://www.aclweb.org/anthology/anthology.bib.gz
and as soon as I have two files, one PDF and one BIB, with the same basename
D14-1162.pdf D14-1162.bib
I know they are connected!
It's so simple, but its impact is immense. Imagine you would have a PDF
D14-1162.pdf
but its bibtex would have a different basename, say
pennington-etal-2014-glove.bib
How on earth would you know they belong together? You would have to resort to web scraping: parse each proceedings HTML page and, for each PDF link on it, find the '.bib' HTML link that is visually 'nearest' to it. This is programming hell.
Having a local paper collection, with papers renamed as above, makes searching (an issue that has been the subject of quite a few postings above) a dream: just list your local papers and pipe the list to a text file. Now use that text file as a "poor man's index" using, say, grep. You can grep it with any regular expression you like
grep -E 'your regexp' index.txt
If you rename your local papers as above, you will be amazed at what you can find by such a simple method!
So thank you for making local collections possible with such genial ideas like providing a cumulative .bib file and using consistent names across the whole site for both papers and their bibliographic information. Forget OAI-PMH, federated repository aggregators and all that! All a truly open access paper repository needs is those two simple things!
lucy3
mjpost
Pranav-Goel
mbollmann
AmyOlex
BramVanroy
AGalassi
akoehn
zixiu-alex-wu
This thread is intended to collect all feedback, suggestions, bug reports, etc. for the new Anthology website in the
static-rewritebranch.(Edit: live demo here at http://aclweb.org/anthology)
If you do not have a GitHub account, you're also welcome to send me feedback via e-mail (marcel@bollmann.me) or Twitter (\@mmbollmann)!
Known Issues