LibraryOfCongress / api.congress.gov

congress.gov API
642 stars 39 forks source link

(amendments API) list returns `updateDate` that differs from details API #112

Closed ryparker closed 8 months ago

ryparker commented 1 year ago

The response from amendment list API looks something like this: https://api.congress.gov/v3/amendment/118?format=json&api_key=*

{
    "amendments": [
       {
            "congress": 118,
            "description": "An amendment numbered 48 printed in House Report 118-142 to prohibit cluster munitions or cluster munitions technology be sold or transferred to Ukraine.",
            "latestAction": {
                "actionDate": "2023-07-13",
                "actionTime": "22:59:37",
                "text": "On agreeing to the Greene (GA) amendment (A028) Failed by recorded vote: 147 - 276, 2 Present (Roll no. 317). "
            },
            "number": "243",
            "type": "HAMDT",
            "updateDate": "2023-07-14T10:01:38Z",
            "url": "https://api.congress.gov/v3/amendment/118/hamdt/243?format=json"
        },
        ....
    ],
    "pagination": {
        "count": 1063,
        "next": "https://api.congress.gov/v3/amendment/118?offset=20&limit=20&format=json"
    },
    "request": {
        "congress": "118",
        "contentType": "application/json",
        "format": "json"
    }
}

Notice the updateDate on HAMDT-243 is 2023-07-14T10:01:38Z. However when I request the details API for that same amendment the updateDate sometimes differs from the list API's value.

Response for amendment details API of HAMDT-243: https://api.congress.gov/v3/amendment/118/HAMDT/243?format=json&api_key=*

{
    "amendment": {
        "actions": {
            "count": 8,
            "url": "https://api.congress.gov/v3/amendment/118/hamdt/243/actions?format=json"
        },
        "amendedBill": {
            "congress": 118,
            "number": "2670",
            "originChamber": "House",
            "originChamberCode": "H",
            "title": "National Defense Authorization Act for Fiscal Year 2024",
            "type": "HR",
            "updateDateIncludingText": "2023-07-15T19:26:16Z",
            "url": "https://api.congress.gov/v3/bill/118/hr/2670?format=json"
        },
        "chamber": "House of Representatives",
        "congress": 118,
        "description": "An amendment numbered 48 printed in House Report 118-142 to prohibit cluster munitions or cluster munitions technology be sold or transferred to Ukraine.",
        "latestAction": {
            "actionDate": "2023-07-13",
            "actionTime": "22:59:37",
            "links": [
                {
                    "name": "Roll no. 317",
                    "url": "https://clerk.house.gov/Votes/2023317"
                }
            ],
            "text": "On agreeing to the Greene (GA) amendment (A028) Failed by recorded vote: 147 - 276, 2 Present (Roll no. 317). "
        },
        "number": "243",
        "sponsors": [
            {
                "bioguideId": "G000596",
                "district": 14,
                "firstName": "Marjorie",
                "fullName": "Rep. Greene, Marjorie Taylor [R-GA-14]",
                "lastName": "Greene",
                "middleName": "Taylor",
                "party": "R",
                "state": "GA",
                "url": "https://api.congress.gov/v3/member/G000596?format=json"
            }
        ],
        "submittedDate": "2023-07-13T04:00:00Z",
        "type": "HAMDT",
        "updateDate": "2023-07-14T10:01:36Z"
    },
    "request": {
        "amendmentNumber": "243",
        "amendmentType": "hamdt",
        "congress": "118",
        "contentType": "application/json",
        "format": "json"
    }
}

Notice how the updateDate from the details API is 2023-07-14T10:01:36Z, however we expected this to match the list API's value of 2023-07-14T10:01:38Z.

Note that this is not the case for all amendments. Some are equal as expected but i've noticed many with this issue.

The reason why this matters to me is because my scraper is caching these amendments based on the updateDate value and when these timestamps differ it causes unnecessary recollection of amendment data that has not actually changed.

ryparker commented 1 year ago

I'm actually noticing that the updateDate on the list API is sometimes wildly different than any dates returned on the details API. See this example of HAMDT-235

The list API response: https://api.congress.gov/v3/amendment/118?format=json&api_key=*

{
    "amendments": [
        ...
        {
            "congress": 118,
            "description": "An amendment numbered 33 printed in House Report 118-142 to eliminate any offices of Diversity, Equity, and Inclusion along with the personnel in said offices within the offices of the Armed Forces and Department of Defense.",
            "latestAction": {
                "actionDate": "2023-07-13",
                "actionTime": "22:42:47",
                "text": "On agreeing to the Norman amendment (A020) Agreed to by recorded vote: 214 - 213 (Roll no. 312). "
            },
            "number": "235",
            "type": "HAMDT",
            "updateDate": "2023-07-15T07:16:22Z",
            "url": "https://api.congress.gov/v3/amendment/118/hamdt/235?format=json"
        },
        ...
    ],
    "pagination": {
        "count": 1063,
        "next": "https://api.congress.gov/v3/amendment/118?offset=20&limit=20&format=json"
    },
    "request": {
        "congress": "118",
        "contentType": "application/json",
        "format": "json"
    }
}

Amendment details API response: https://api.congress.gov/v3/amendment/118/HAMDT/235?format=json&api_key=*

{
    "amendment": {
        "actions": {
            "count": 13,
            "url": "https://api.congress.gov/v3/amendment/118/hamdt/235/actions?format=json"
        },
        "amendedBill": {
            "congress": 118,
            "number": "2670",
            "originChamber": "House",
            "originChamberCode": "H",
            "title": "National Defense Authorization Act for Fiscal Year 2024",
            "type": "HR",
            "updateDateIncludingText": "2023-07-15T19:26:16Z",
            "url": "https://api.congress.gov/v3/bill/118/hr/2670?format=json"
        },
        "chamber": "House of Representatives",
        "congress": 118,
        "description": "An amendment numbered 33 printed in House Report 118-142 to eliminate any offices of Diversity, Equity, and Inclusion along with the personnel in said offices within the offices of the Armed Forces and Department of Defense.",
        "latestAction": {
            "actionDate": "2023-07-13",
            "actionTime": "22:42:47",
            "links": [
                {
                    "name": "Roll no. 312",
                    "url": "https://clerk.house.gov/Votes/2023312"
                }
            ],
            "text": "On agreeing to the Norman amendment (A020) Agreed to by recorded vote: 214 - 213 (Roll no. 312). "
        },
        "number": "235",
        "sponsors": [
            {
                "bioguideId": "N000190",
                "district": 5,
                "firstName": "Ralph",
                "fullName": "Rep. Norman, Ralph [R-SC-5]",
                "lastName": "Norman",
                "party": "R",
                "state": "SC",
                "url": "https://api.congress.gov/v3/member/N000190?format=json"
            }
        ],
        "submittedDate": "2023-07-13T04:00:00Z",
        "type": "HAMDT",
        "updateDate": "2023-07-14T10:01:38Z"
    },
    "request": {
        "amendmentNumber": "235",
        "amendmentType": "hamdt",
        "congress": "118",
        "contentType": "application/json",
        "format": "json"
    }
}

As you can see the list API returns an updateDate of 2023-07-15T07:16:22Z however that timestamp is not even close to any timestamp in the details response.

mnewatloc commented 1 year ago

We are investigating this. Thank you for bringing it to our attention.

ryparker commented 8 months ago

Seems like this has been fixed.