search

MicheleCotrufo / pdf2doi

A python library/command-line tool to extract the DOI or other identifiers of a scientific paper from a pdf file.
101 stars 18 forks source link

Not a bug, just results have 'false-positives'? (Test scenario) #10

Closed 192933488S closed 2 years ago

192933488S commented 2 years ago

Hello, this is a very nifty tool, but I get the following results on a test case I set up containing a bundle of random PDFs from my library.

[pdf2doi]: ................
DOI             10.1109/MS.2018.2141038                  ./10.1109@MS.2018.2141038.pdf

DOI             10.1145/3341227                          ./10.1145@3341227 MUST and MUST NOT.pdf

DOI             10.1145/38807.38824                      ./120158- Use Case Template-20160821_0954877.pdf

DOI             10.1016/j.jss.2016.02.047                ./120216- Software Requirements Specification Template-20160821_0951179.pdf

DOI             10.1007/978-3-319-09816-6                ./2014_Book_Autonomy Requirements Engineering for Space Missions NASA Springer.pdf

DOI             10.1007/978-1-4614-5377-2                ./293233main_62651main_1_pmchallenge_hraster.pdf

The first answer is pretty cool, extracted from filename. The 1st, 2nd and 5th are correct. The rest is false. Specifically the last one is close to target, but I am yet about to understand how. The file is a presentation, without mentioning the extracted DOI, but has similar contents.

Sincerely

MicheleCotrufo commented 2 years ago

Thanks for your feedback! Indeed, it is possible to have "false positives". This tool tries several methods to identify the DOI, first in conventional ways (i.e. looking into the pdf metadata, or looking for a valid DOI in the pdf title or text). If these methods do not succeed, it tries more complicated things, such as extracting portions of text from the pdf file, googling them, and then looking for DOIs in the results of the google search. Thus, if your pdf file contains some text which can be somehow associated with a published paper via a google result, that will likely happen. In general, there is not a standard way to understand if a pdf file is a scientific publication or anything else, so I recommend running pdf2doi only on pdf files that contain scientific publications.

If you make the output more verbose (with the optional command -v), you can have more info about what pdf2doi is doing for each pdf file, and figure out why that specific DOI was associated with that file. If you are running it again on the same files, first make sure to clear the pdf metadata, because during the previous call to pdf2doi the found DOIs were stored in the pdf metadata. In summary, run the commands

pdf2doi "path/to/folder"  -v -id ""
pdf2doi "path/to/folder"  -v

The first command will replace any DOI previously stored by pdf2doi in the file metadata by "". See also https://github.com/MicheleCotrufo/pdf2doi#manually-associate-the-correct-identifier-to-a-file-from-command-line and the third-to-last paragraph here https://github.com/MicheleCotrufo/pdf2doi#description.