search

EleutherAI / the-pile

MIT License
1.46k stars 126 forks source link

USPTO Patent #26

Closed cfoster0 closed 3 years ago

cfoster0 commented 3 years ago

Language: English Date ranges: 1976-2000 (APS), 2001-2020 (XML) Size: > 10GB.

Could either do patent applications or patent grants. Most relevant sections are likely "Background", followed by "Summary" / "Brief Description" and then "Detailed Description". May want to exclude claims and discussion of drawings.

https://bulkdata.uspto.gov/

StellaAthena commented 3 years ago

This looks great to me! I don’t think we’ve done any APS files, so that might require writing a new processing script.

@cfoster0 do you want to claim responsibility for this?

cfoster0 commented 3 years ago

@StellaAthena Sure thing! I'll give it a try.

cfoster0 commented 3 years ago

Based on discussions today on Discord, I've proceeded on this for patent grants, specifically the "Background" sections. The majority of the work is done: the main outstanding items are deciding what metadata to include and what additional formatting (if any) is necessary.

cfoster0 commented 3 years ago

Script is available now here. My estimate is the total text size (not including metadata) is approximately 23 GiB.

cfoster0 commented 3 years ago

Pre-processed data is available here for now.

Dataset info:

COMPRESSED ARCHIVE SIZE:
10.983731 GiB

UNCOMPRESSED TEXT SIZE:
22.882614 GiB

DOCUMENT COUNT:
5,883,037

MEAN DOCUMENT SIZE:
4.17641767 kilobytes

EXAMPLE TEXT:
This invention relates in general to solvents for use in solubilizing dyes and, more particularly, to solvents used in the production of carbonless paper, microcapsules containing a chromogenic compound solubilized by such solvents, and pressure sensitive marking and recording material coated with such microcapsules. The invention also relates to a process for making carbonless paper solvents.
Carbonless paper and other marking systems depend upon localized contact between a chromogenic compound such as a leuco dye and a color developing substance to produce a visible indicia ...

EXAMPLE METADATA:
{'bibliographic_information': {'Patent Number': '053858794', 'Series Code': '7', 'Application Number': '9662952', 'Application Type': '1', 'Art unit': '115', 'Application Filing Date': '19921026', 'Title of Invention': 'Carbonless paper solvent comprising diisopropylmethylnaphthalene and products utilizing same', 'Issue Date': '19950131', 'Number of Claims': '2', 'Exemplary Claim Number(s)': '1', 'Assistant Examiner': 'Einsmann; Margaret', 'Primary Examiner': 'Lieberman; Paul'}, ... }