LibraryOfCongress / api.congress.gov

congress.gov API
600 stars 38 forks source link

Bills' action dates/update dates not within fromDateTime - toDateTime range #261

Open sdxiang opened 1 week ago

sdxiang commented 1 week ago

The call

https://api.congress.gov/v3/bill/117?fromDateTime=2022-01-01T00:00:00Z&toDateTime=2022-01-31T23:59:59Z

returns 14 bills, but only one of them has an update date within the time range, and none of them has a latest action date within the time range. Not sure if this is a bug or if I'm just misunderstanding what it's filtering on?

apreiter18 commented 1 week ago

Hi @sdxiang - the <updateDate> element is thedate of update on Congress.gov. This update date does not include updates to bill text. The <updateDate> is the date of the last update received for the legislative entity. It’s not a date corresponding to the legislative date or legislative action date. Hope this helps!

sdxiang commented 1 week ago

I think I'm still confused as to what date the API is filtering on-- for example, here is a bill returned by the original call, and there is no date within the specified time range (2022-01-01 through 2022-01-31). image I took a look at some of the bill endpoints linked in the response, and found that in most cases the only date within the time range is actually the first update date under "titles" image What does this date mean?

apreiter18 commented 1 week ago

Hi @sdxiang - please confirm or correct I am understanding you question correctly. When I perform this call https://api.congress.gov/v3/bill/117?fromDateTime=2022-01-01T00:00:00Z&toDateTime=2022-01-31T23:59:59Z&api_key=INSERT_KEY I get a list of bills that have <updateDate> elements within the time frame queried:

For example, SRes 469: image

The <updateDate> is the date of the last update received for the legislative entity. It’s not a date corresponding to the legislative date or legislative action date. The data partner can update records and resend the information to Congress.gov.

HOWEVER, I do see your point, though, for certain bills not having a corresponding <updateDate> in the list level. For example, S 3206 has a 2023 date in the list-level but a 2022 date in the detail level (see: https://api.congress.gov/v3/bill/117/s/3206?format=xml&api_key=INSERT_KEY), which might explain why it is being returned in the query.

I will submit a ticket to investigate this discrepancy. Thanks.

sdxiang commented 1 week ago

While the first bill returned does have an <updateDate> within the time frame queried, many of them do not, for example:

S 3206 image

SRES 443 image

S 4 image

I think you're right that they're being returned because they have Jan 2022 update dates in the detail level but not the list level, and this is what's causing the confusion.

apreiter18 commented 1 week ago

Ok a quick update as I investigate:

Currently, the list level (e.g., https://api.congress.gov/v3/bill/117?fromDateTime=2022-01-01T00:00:00Z&toDateTime=2022-01-31T23:59:59Z&api_key=DEMO_KEY) is returning the updateDateIncludingText date as just the in the list level. HOWEVER, users like this one, can query by which is returning any bill with a 2022 update date even if it is NOT present in the list level.

For example, 117 S 3206 has this update date in the list level: 2023-01-02

When I go look at the record in the detail level (https://api.congress.gov/v3/bill/117/s/3206?format=xml&api_key=DEMO_KEY) I see two dates:

image

So what I think is happening for these bills is the list level <updateDate>= <updateDateIncludingText> in the detail level, which would explain why the bill is still being returned in the list level query you shared.

Thank you for reporting this, @sdxiang - we will investigate further and updates will be posted in the change log.

sdxiang commented 5 days ago

Just wanted to flag that this list-level vs. detail-level updateDate issue seems to be affecting other endpoints as well:

The API documentation for the bound-congressional-record endpoint says that the response to https://api.congress.gov/v3/bound-congressional-record? should be sorted by most recent. However, when I make this query, the response does not seem to be sorted by either of the dates exposed (date or updateDate).

image

When I take a look at the individual records linked in the list response (e.g. https://api.congress.gov/v3/bound-congressional-record/1943/1/6?), it seems like some components have more recent update dates than others. For this example, the appendix has an updateDate of 2024-03-28 while the other sections have updateDates of 2024-04-17, which is more consistent with this record's placement in the list response, but 2024-03-28 is the updateDate exposed for the entire record.

ryparker commented 5 days ago

@sdxiang I've reported the sorting issue here: https://github.com/LibraryOfCongress/api.congress.gov/issues/223