Closed cfoster0 closed 3 years ago
This looks great to me! I don’t think we’ve done any APS files, so that might require writing a new processing script.
@cfoster0 do you want to claim responsibility for this?
@StellaAthena Sure thing! I'll give it a try.
Based on discussions today on Discord, I've proceeded on this for patent grants, specifically the "Background" sections. The majority of the work is done: the main outstanding items are deciding what metadata to include and what additional formatting (if any) is necessary.
Script is available now here. My estimate is the total text size (not including metadata) is approximately 23 GiB.
Pre-processed data is available here for now.
Dataset info:
COMPRESSED ARCHIVE SIZE:
10.983731 GiB
UNCOMPRESSED TEXT SIZE:
22.882614 GiB
DOCUMENT COUNT:
5,883,037
MEAN DOCUMENT SIZE:
4.17641767 kilobytes
EXAMPLE TEXT:
This invention relates in general to solvents for use in solubilizing dyes and, more particularly, to solvents used in the production of carbonless paper, microcapsules containing a chromogenic compound solubilized by such solvents, and pressure sensitive marking and recording material coated with such microcapsules. The invention also relates to a process for making carbonless paper solvents.
Carbonless paper and other marking systems depend upon localized contact between a chromogenic compound such as a leuco dye and a color developing substance to produce a visible indicia ...
EXAMPLE METADATA:
{'bibliographic_information': {'Patent Number': '053858794', 'Series Code': '7', 'Application Number': '9662952', 'Application Type': '1', 'Art unit': '115', 'Application Filing Date': '19921026', 'Title of Invention': 'Carbonless paper solvent comprising diisopropylmethylnaphthalene and products utilizing same', 'Issue Date': '19950131', 'Number of Claims': '2', 'Exemplary Claim Number(s)': '1', 'Assistant Examiner': 'Einsmann; Margaret', 'Primary Examiner': 'Lieberman; Paul'}, ... }
Language: English Date ranges: 1976-2000 (APS), 2001-2020 (XML) Size: > 10GB.
Could either do patent applications or patent grants. Most relevant sections are likely "Background", followed by "Summary" / "Brief Description" and then "Detailed Description". May want to exclude claims and discussion of drawings.
https://bulkdata.uspto.gov/