search

usgpo / bill-status

Information about Bill Status XML Bulk Data including the XML User Guide.
https://www.govinfo.gov/bulkdata/BILLSTATUS
152 stars 64 forks source link

(<textVersions>): formats missing in some legislation #232

Open ryparker opened 4 months ago

ryparker commented 4 months ago

Some bill-status return <textVersion> items that have empty <formats> tags, however this data is available on the congress.gov API and the actual format files (pdfs,xml,html) are available from GPO.

Example request:

curl --location 'https://govinfo.gov/bulkdata/BILLSTATUS/118/hr/BILLSTATUS-118hr3496.xml'

Response:

<?xml version="1.0" encoding="utf-8" standalone="no"?>
<billStatus>
    <version>3.0.0</version>
    <bill>
        <number>3496</number>
        <updateDate>2024-02-14T19:25:09Z</updateDate>
        <updateDateIncludingText>2024-02-14T19:25:09Z</updateDateIncludingText>
        <originChamber>House</originChamber>
        <originChamberCode>H</originChamberCode>
        <type>HR</type>
        <introducedDate>2023-05-18</introducedDate>
        <congress>118</congress>
        <constitutionalAuthorityStatementText>
            <![CDATA[<pre>[Congressional Record Volume 169, Number 84 (Thursday, May 18, 2023)][House]From the Congressional Record Online through the Government Publishing Office [<a href="https://www.gpo.gov">www.gpo.gov</a>]By Ms. HAGEMAN:H.R. 3496.Congress has the power to enact this legislation pursuantto the following:Article I Section 8The single subject of this legislation is:To make technical amendments to update statutory referencesto certain provisions which were formerly classified tochapters 14 and 19 of title 25, United States Code, and tocorrect related technical errors[Page H2458]</pre>]]>
        </constitutionalAuthorityStatementText>
        <committees>
            <item>
                <systemCode>ssju00</systemCode>
                <name>Judiciary Committee</name>
                <chamber>Senate</chamber>
                <type>Standing</type>
                <activities>
                    <item>
                        <name>Referred to</name>
                        <date>2023-11-14T23:31:52Z</date>
                    </item>
                </activities>
            </item>
            <item>
                <systemCode>hsju00</systemCode>
                <name>Judiciary Committee</name>
                <chamber>House</chamber>
                <type>Standing</type>
                <activities>
                    <item>
                        <name>Reported by</name>
                        <date>2023-09-29T21:23:46Z</date>
                    </item>
                    <item>
                        <name>Markup by</name>
                        <date>2023-05-24T15:58:44Z</date>
                    </item>
                    <item>
                        <name>Referred to</name>
                        <date>2023-05-18T13:09:10Z</date>
                    </item>
                </activities>
            </item>
        </committees>
        <committeeReports>
            <committeeReport>
                <citation>H. Rept. 118-235</citation>
            </committeeReport>
        </committeeReports>
        <relatedBills>
            <item>
                <title>SUPPORT for Patients and Communities Reauthorization Act</title>
                <congress>118</congress>
                <number>3393</number>
                <type>S</type>
                <latestAction>
                    <actionDate>2024-02-01</actionDate>
                    <text>Placed on Senate Legislative Calendar under General Orders. Calendar No. 319.</text>
                </latestAction>
                <relationshipDetails>
                    <item>
                        <type>Related bill</type>
                        <identifiedBy>CRS</identifiedBy>
                    </item>
                </relationshipDetails>
            </item>
        </relatedBills>
        <actions>
            <item>
                <actionDate>2023-11-14</actionDate>
                <committees>
                    <item>
                        <systemCode>ssju00</systemCode>
                        <name>Judiciary Committee</name>
                    </item>
                </committees>
                <sourceSystem>
                    <name>Senate</name>
                </sourceSystem>
                <text>Received in the Senate and Read twice and referred to the Committee on the Judiciary.</text>
                <type>IntroReferral</type>
            </item>
            <item>
                <actionDate>2023-11-13</actionDate>
                <actionTime>17:02:01</actionTime>
                <text>Motion to reconsider laid on the table Agreed to without objection.</text>
                <type>Floor</type>
                <actionCode>H38310</actionCode>
                <sourceSystem>
                    <code>2</code>
                    <name>House floor actions</name>
                </sourceSystem>
            </item>
            <item>
                <actionDate>2023-11-13</actionDate>
                <actionTime>17:01:56</actionTime>
                <text>On motion to suspend the rules and pass the bill Agreed to by voice vote. (text: CR H5679-5687)</text>
                <type>Floor</type>
                <actionCode>H37300</actionCode>
                <sourceSystem>
                    <code>2</code>
                    <name>House floor actions</name>
                </sourceSystem>
            </item>
            <item>
                <actionDate>2023-11-13</actionDate>
                <actionTime>17:01:56</actionTime>
                <text>Passed/agreed to in House: On motion to suspend the rules and pass the bill Agreed to by voice vote.(text: CR H5679-5687)</text>
                <type>Floor</type>
                <actionCode>8000</actionCode>
                <sourceSystem>
                    <code>9</code>
                    <name>Library of Congress</name>
                </sourceSystem>
            </item>
            <item>
                <actionDate>2023-11-13</actionDate>
                <actionTime>16:56:10</actionTime>
                <text>DEBATE - The House proceeded with forty minutes of debate on H.R. 3496.</text>
                <type>Floor</type>
                <actionCode>H8D000</actionCode>
                <sourceSystem>
                    <code>2</code>
                    <name>House floor actions</name>
                </sourceSystem>
            </item>
            <item>
                <actionDate>2023-11-13</actionDate>
                <actionTime>16:56:08</actionTime>
                <text>Considered under suspension of the rules. (consideration: CR H5679-5688)</text>
                <type>Floor</type>
                <actionCode>H30000</actionCode>
                <sourceSystem>
                    <code>2</code>
                    <name>House floor actions</name>
                </sourceSystem>
            </item>
            <item>
                <actionDate>2023-11-13</actionDate>
                <actionTime>16:55:18</actionTime>
                <text>Mr. Cline moved to suspend the rules and pass the bill.</text>
                <type>Floor</type>
                <actionCode>H30300</actionCode>
                <sourceSystem>
                    <code>2</code>
                    <name>House floor actions</name>
                </sourceSystem>
            </item>
            <item>
                <actionDate>2023-09-29</actionDate>
                <text>Placed on the House Calendar, Calendar No. 43.</text>
                <type>Calendars</type>
                <actionCode>H12420</actionCode>
                <sourceSystem>
                    <code>2</code>
                    <name>House floor actions</name>
                </sourceSystem>
                <calendarNumber>
                    <calendar>H00043</calendar>
                </calendarNumber>
            </item>
            <item>
                <actionDate>2023-09-29</actionDate>
                <text>Reported by the Committee on Judiciary. H. Rept. 118-235.</text>
                <type>Committee</type>
                <actionCode>H12200</actionCode>
                <sourceSystem>
                    <code>2</code>
                    <name>House floor actions</name>
                </sourceSystem>
                <committees>
                    <item>
                        <systemCode>hsju00</systemCode>
                        <name>Judiciary Committee</name>
                    </item>
                </committees>
            </item>
            <item>
                <actionDate>2023-09-29</actionDate>
                <text>Reported by the Committee on Judiciary. H. Rept. 118-235.</text>
                <type>Committee</type>
                <actionCode>5000</actionCode>
                <sourceSystem>
                    <code>9</code>
                    <name>Library of Congress</name>
                </sourceSystem>
                <committees>
                    <item>
                        <systemCode>hsju00</systemCode>
                        <name>Judiciary Committee</name>
                    </item>
                </committees>
            </item>
            <item>
                <actionDate>2023-05-24</actionDate>
                <committees>
                    <item>
                        <systemCode>hsju00</systemCode>
                        <name>Judiciary Committee</name>
                    </item>
                </committees>
                <sourceSystem>
                    <code>1</code>
                    <name>House committee actions</name>
                </sourceSystem>
                <text>Ordered to be Reported by Voice Vote.</text>
                <type>Committee</type>
            </item>
            <item>
                <actionDate>2023-05-24</actionDate>
                <committees>
                    <item>
                        <systemCode>hsju00</systemCode>
                        <name>Judiciary Committee</name>
                    </item>
                </committees>
                <sourceSystem>
                    <code>1</code>
                    <name>House committee actions</name>
                </sourceSystem>
                <text>Committee Consideration and Mark-up Session Held.</text>
                <type>Committee</type>
            </item>
            <item>
                <actionDate>2023-05-18</actionDate>
                <text>Referred to the House Committee on the Judiciary.</text>
                <type>IntroReferral</type>
                <actionCode>H11100</actionCode>
                <sourceSystem>
                    <code>2</code>
                    <name>House floor actions</name>
                </sourceSystem>
                <committees>
                    <item>
                        <systemCode>hsju00</systemCode>
                        <name>Judiciary Committee</name>
                    </item>
                </committees>
            </item>
            <item>
                <actionDate>2023-05-18</actionDate>
                <text>Introduced in House</text>
                <type>IntroReferral</type>
                <actionCode>Intro-H</actionCode>
                <sourceSystem>
                    <code>9</code>
                    <name>Library of Congress</name>
                </sourceSystem>
            </item>
            <item>
                <actionDate>2023-05-18</actionDate>
                <text>Introduced in House</text>
                <type>IntroReferral</type>
                <actionCode>1000</actionCode>
                <sourceSystem>
                    <code>9</code>
                    <name>Library of Congress</name>
                </sourceSystem>
            </item>
        </actions>
        <sponsors>
            <item>
                <bioguideId>H001096</bioguideId>
                <fullName>Rep. Hageman, Harriet M. [R-WY-At Large]</fullName>
                <firstName>Harriet</firstName>
                <lastName>Hageman</lastName>
                <party>R</party>
                <state>WY</state>
                <middleName>M.</middleName>
                <isByRequest>N</isByRequest>
            </item>
        </sponsors>
        <cboCostEstimates>
            <item>
                <pubDate>2023-06-27T17:58:00Z</pubDate>
                <title>H.R. 3496, a bill to make technical amendments to update statutory references to certain provisions which were formerly classified to chapters 14 and 19 of title 25, United States Code, and to correct related technical errors</title>
                <url>https://www.cbo.gov/publication/59322</url>
                <description>As ordered reported by the House Committee on the Judiciary onMay 24, 2023</description>
            </item>
        </cboCostEstimates>
        <policyArea>
            <name>Native Americans</name>
        </policyArea>
        <subjects>
            <legislativeSubjects>
                <item>
                    <name>Congressional operations and organization</name>
                </item>
                <item>
                    <name>Indian claims</name>
                </item>
                <item>
                    <name>Indian lands and resources rights</name>
                </item>
            </legislativeSubjects>
            <policyArea>
                <name>Native Americans</name>
            </policyArea>
        </subjects>
        <summaries>
            <summary>
                <versionCode>00</versionCode>
                <actionDate>2023-05-18</actionDate>
                <actionDesc>Introduced in House</actionDesc>
                <updateDate>2023-11-07T16:30:13Z</updateDate>
                <text>
                    <![CDATA[ <p>This bill updates references in the <em>U.S. Code</em> to certain provisions in Title 25 (Indians). In 2016, Congress transferred certain provisions in Chapter 14 (Miscellaneous) and Chapter 19 (Indian Land Claims Settlements) of Title 25 to new chapters at the end of the title as part of an effort to reclassify the code. To reflect the reclassification of the code, this bill updates references to Title 25.</p>]]>
                </text>
            </summary>
            <summary>
                <versionCode>07</versionCode>
                <actionDate>2023-09-29</actionDate>
                <actionDesc>Reported to House</actionDesc>
                <updateDate>2023-11-13T15:19:11Z</updateDate>
                <text>
                    <![CDATA[ <p>This bill updates references in the <em>U.S. Code</em> to certain provisions in Title 25 (Indians). In 2016, Congress transferred certain provisions in Chapter 14 (Miscellaneous) and Chapter 19 (Indian Land Claims Settlements) of Title 25 to new chapters at the end of the title as part of an effort to reclassify the code. To reflect the reclassification of the code, this bill updates references to Title 25.</p>]]>
                </text>
            </summary>
            <summary>
                <versionCode>53</versionCode>
                <actionDate>2023-11-13</actionDate>
                <actionDesc>Passed House</actionDesc>
                <updateDate>2023-11-20T17:49:06Z</updateDate>
                <text>
                    <![CDATA[ <p>This bill updates references in the <em>U.S. Code</em> to certain provisions in Title 25 (Indians). In 2016, Congress transferred certain provisions in Chapter 14 (Miscellaneous) and Chapter 19 (Indian Land Claims Settlements) of Title 25 to new chapters at the end of the title as part of an effort to reclassify the code. To reflect the reclassification of the code, this bill updates references to Title 25.</p>]]>
                </text>
            </summary>
        </summaries>
        <title>To make technical amendments to update statutory references to certain provisions which were formerly classified to chapters 14 and 19 of title 25, United States Code, and to correct related technical errors.</title>
        <titles>
            <item>
                <titleType>Display Title</titleType>
                <title>To make technical amendments to update statutory references to certain provisions which were formerly classified to chapters 14 and 19 of title 25, United States Code, and to correct related technical errors.</title>
            </item>
            <item>
                <titleType>Official Title as Introduced</titleType>
                <title>To make technical amendments to update statutory references to certain provisions which were formerly classified to chapters 14 and 19 of title 25, United States Code, and to correct related technical errors.</title>
                <billTextVersionName>Introduced in House</billTextVersionName>
                <billTextVersionCode>IH</billTextVersionCode>
            </item>
        </titles>
        <textVersions>
            <item>
                <type>Referred in Senate</type>
                <date>2023-11-14T05:00:00Z</date>
                <formats/>
            </item>
            <item>
                <type>Engrossed in House</type>
                <date>2023-11-13T05:00:00Z</date>
                <formats/>
            </item>
            <item>
                <type>Reported in House</type>
                <date>2023-09-29T04:00:00Z</date>
                <formats/>
            </item>
            <item>
                <type>Introduced in House</type>
                <date>2023-05-18T04:00:00Z</date>
                <formats/>
            </item>
        </textVersions>
        <latestAction>
            <actionDate>2023-11-14</actionDate>
            <text>Received in the Senate and Read twice and referred to the Committee on the Judiciary.</text>
        </latestAction>
    </bill>
    <dublinCore xmlns:dc="http://purl.org/dc/elements/1.1/">
        <dc:format>text/xml</dc:format>
        <dc:language>EN</dc:language>
        <dc:rights>Pursuant to Title 17 Section 105 of the United States Code, this file is not subject to copyright protection and is in the public domain.</dc:rights>
        <dc:contributor>Congressional Research Service, Library of Congress</dc:contributor>
        <dc:description>This file contains bill summaries and statuses for federal legislation. A bill summary describes the most significant provisions of a piece of legislation and details the effects the legislative text may have on current law and federal programs. Bill summaries are authored by the Congressional Research Service (CRS) of the Library of Congress. As stated in Public Law 91-510 (2 USC 166 (d)(6)), one of the duties of CRS is "to prepare summaries and digests of bills and resolutions of a public general nature introduced in the Senate or House of Representatives". For more information, refer to the User Guide that accompanies this file.</dc:description>
    </dublinCore>
</billStatus>

Notice how <textVersions> has empty <formats/> tags:

  <textVersions>
            <item>
                <type>Referred in Senate</type>
                <date>2023-11-14T05:00:00Z</date>
                <formats/>
            </item>
            <item>
                <type>Engrossed in House</type>
                <date>2023-11-13T05:00:00Z</date>
                <formats/>
            </item>
            <item>
                <type>Reported in House</type>
                <date>2023-09-29T04:00:00Z</date>
                <formats/>
            </item>
            <item>
                <type>Introduced in House</type>
                <date>2023-05-18T04:00:00Z</date>
                <formats/>
            </item>
        </textVersions>

However if you check congress.gov's API you'll see the formats are available:

Request:

curl --location 'https://api.congress.gov/v3/bill/118/hr/3496/text?format=json&api_key=<API_KEY>'

Response:

{
    "pagination": {
        "count": 4
    },
    "request": {
        "billNumber": "3496",
        "billType": "hr",
        "billUrl": "https://api.congress.gov/v3/bill/118/hr/3496?format=json",
        "congress": "118",
        "contentType": "application/json",
        "format": "json"
    },
    "textVersions": [
        {
            "date": "2023-11-14T05:00:00Z",
            "formats": [
                {
                    "type": "Formatted Text",
                    "url": "https://www.congress.gov/118/bills/hr3496/BILLS-118hr3496rfs.htm"
                },
                {
                    "type": "PDF",
                    "url": "https://www.congress.gov/118/bills/hr3496/BILLS-118hr3496rfs.pdf"
                }
            ],
            "type": "Referred in Senate"
        },
        {
            "date": "2023-11-13T05:00:00Z",
            "formats": [
                {
                    "type": "Formatted Text",
                    "url": "https://www.congress.gov/118/bills/hr3496/BILLS-118hr3496eh.htm"
                },
                {
                    "type": "PDF",
                    "url": "https://www.congress.gov/118/bills/hr3496/BILLS-118hr3496eh.pdf"
                }
            ],
            "type": "Engrossed in House"
        },
        {
            "date": "2023-09-29T04:00:00Z",
            "formats": [
                {
                    "type": "Formatted Text",
                    "url": "https://www.congress.gov/118/bills/hr3496/BILLS-118hr3496rh.htm"
                },
                {
                    "type": "PDF",
                    "url": "https://www.congress.gov/118/bills/hr3496/BILLS-118hr3496rh.pdf"
                }
            ],
            "type": "Reported in House"
        },
        {
            "date": "2023-05-18T04:00:00Z",
            "formats": [
                {
                    "type": "Formatted Text",
                    "url": "https://www.congress.gov/118/bills/hr3496/BILLS-118hr3496ih.htm"
                },
                {
                    "type": "PDF",
                    "url": "https://www.congress.gov/118/bills/hr3496/BILLS-118hr3496ih.pdf"
                }
            ],
            "type": "Introduced in House"
        }
    ]
}

And by assuming the GPO url you'll notice that the formats are available:

curl --location 'https://www.govinfo.gov/content/pkg/BILLS-118hr3496ih/html/BILLS-118hr3496ih.htm'

I'll provide a list of the affected legislation here:

118th congress

jonquandt commented 4 months ago

Good afternoon,

The BILLSTATUS xml will only return textVersion format information if the xml is included. In the example you cited above, there is no xml available. We do not currently provide links to the pdf or html within the BILLSTATUS xml on the bulkdata repository.

That being said, you may be interested in using our related service to find the related bills and download any relevant information. For example,

https://api.govinfo.gov/related/BILLSTATUS-118hr3496?api_key=DEMO_KEY returns a list of relationships - in this case, Congressional Bills (BILLS), History of Bills (HOB), and Congressional Reports (CRPT)

{
    "relationships": [
        {
            "relationshipLink": "https://api.govinfo.gov/related/BILLSTATUS-118hr3496/BILLS",
            "collection": "BILLS",
            "relationship": "Bill versions"
        },
        {
            "relationshipLink": "https://api.govinfo.gov/related/BILLSTATUS-118hr3496/HOB",
            "collection": "HOB",
            "relationship": "Bill History"
        },
        {
            "relationshipLink": "https://api.govinfo.gov/related/BILLSTATUS-118hr3496/CRPT",
            "collection": "CRPT",
            "relationship": "Congressional Reports"
        }
    ],
    "relatedId": "BILLSTATUS-118hr3496"
}

if you follow the BILLS relationship, you will see a list of different bill versions related to this BILLSTATUS:

{
    "results": [
        {
            "dateIssued": "2023-05-18",
            "billVersion": "ih",
            "packageId": "BILLS-118hr3496ih",
            "packageLink": "https://api.govinfo.gov/packages/BILLS-118hr3496ih/summary",
            "billVersionLabel": "Introduced in House",
            "lastModified": "2023-08-07T13:09:13Z"
        },
        {
            "dateIssued": "2023-09-29",
            "billVersion": "rh",
            "packageId": "BILLS-118hr3496rh",
            "packageLink": "https://api.govinfo.gov/packages/BILLS-118hr3496rh/summary",
            "billVersionLabel": "Reported in House",
            "lastModified": "2023-11-14T05:54:43Z"
        },
        {
            "dateIssued": "2023-11-13",
            "billVersion": "eh",
            "packageId": "BILLS-118hr3496eh",
            "packageLink": "https://api.govinfo.gov/packages/BILLS-118hr3496eh/summary",
            "billVersionLabel": "Engrossed in House",
            "lastModified": "2023-11-15T04:33:15Z"
        },
        {
            "dateIssued": "2023-11-14",
            "billVersion": "rfs",
            "packageId": "BILLS-118hr3496rfs",
            "packageLink": "https://api.govinfo.gov/packages/BILLS-118hr3496rfs/summary",
            "billVersionLabel": "Referred in Senate",
            "lastModified": "2023-11-16T01:54:49Z"
        }
    ],
    "relatedId": "BILLSTATUS-118hr3496"
}

Following any of these will return a summary of information about this content, including download links, like BILLS-118hr3496rfs

{
    "originChamber": "HOUSE",
    "references": [**COLLAPSED FOR READABILITY**],
    "congress": "118",
    "session": "1",
    "detailsLink": "https://www.govinfo.gov/app/details/BILLS-118hr3496rfs",
    "isPrivate": "false",
    "title": "An act h.R. 3496 (RFS) - Referred in Senate",
    "branch": "legislative",
    "isAppropriation": "false",
    "collectionName": "Congressional Bills",
    "download": {
        "premisLink": "https://api.govinfo.gov/packages/BILLS-118hr3496rfs/premis",
        "txtLink": "https://api.govinfo.gov/packages/BILLS-118hr3496rfs/htm",
        "zipLink": "https://api.govinfo.gov/packages/BILLS-118hr3496rfs/zip",
        "modsLink": "https://api.govinfo.gov/packages/BILLS-118hr3496rfs/mods",
        "pdfLink": "https://api.govinfo.gov/packages/BILLS-118hr3496rfs/pdf"
    },
    "pages": "37",
    "related": {"billStatusLink": "https://www.govinfo.gov/bulkdata/BILLSTATUS/118/hr/BILLSTATUS-118hr3496.xml"},
    "relatedLink": "https://api.govinfo.gov/related/BILLS-118hr3496rfs",
    "suDocClassNumber": "Y 1.6:, Y 1.4/6:",
    "dateIssued": "2023-11-14",
    "currentChamber": "SENATE",
    "billVersion": "rfs",
    "billType": "hr",
    "packageId": "BILLS-118hr3496rfs",
    "committees": [{
        "authorityId": "ssju00",
        "chamber": "S",
        "committeeName": "Committee on the Judiciary",
        "type": "S"
    }],
    "collectionCode": "BILLS",
    "governmentAuthor2": "Senate",
    "governmentAuthor1": "Congress",
    "publisher": "U.S. Government Publishing Office",
    "docClass": "hr",
    "lastModified": "2023-11-16T01:54:49Z",
    "category": "Bills and Statutes",
    "billNumber": "3496",
    "otherIdentifier": {
        "migrated-doc-id": "f:h3496rfs.txt",
        "parent-ils-system-id": "000501532",
        "child-ils-title": "House bills",
        "parent-ils-title": "Congressional bills",
        "child-ils-system-id": "000325573",
        "stock-number": "021-610-00252-9"
    }
}
ryparker commented 3 months ago

Thanks for the workaround Jon.

Does it make sense to keep this open as a feature request? It would be nice have the non-xml format URLs included in the BILLSTATUS response.