Improve the general purpose citation detector for obviously missing citations

lmullen / legal-modernism

Law and legal practice modernized in the nineteenth-century United States. We are studying and visualizing the history of the modernization of American law.

https://legalmodernism.org

MIT License

4 stars 0 forks source link

Improve the general purpose citation detector for obviously missing citations #40

Closed lmullen closed 2 years ago

lmullen commented 2 years ago

[x] Clean up OCR errors. (This is #36, but a non-trivial number part of the problem was OCR errors.)
[x] Modify regex to include commas as well as periods at the end of the reporter abbreviation.

lmullen commented 2 years ago

Pomeroy-citations-RA-check.csv

This shows the problems in a particular treatise.

lmullen commented 2 years ago

@kfunk074 In this CSV, there are a number of rows labeled "missing multiple page numbers." E.g., the raw OCR text will have 16 Barb. 825, 830. (Ignore for now any OCR problems with the numerals.) The spreadsheet complains that 16 Barb. 825 is detected but not 16 Barb. 830. I think that is a mistake on the part of the compiler: we only care about the first page number in the case, and the , 830 can be safely ignored, as could the hypothetical -830 in 16 Barb. 825-830 or its ilk. Correct me if I am wrong.

lmullen commented 2 years ago

Assuming my comment is correct, the PR above fixes all of the problems identified by the RA, except for OCR errors which are dealt with separately.