data-liberation-project / aphis-inspection-reports

Inspection data and PDFs from the USDA's Animal and Plant Health Inspection Service.
13 stars 3 forks source link

Refactor RSS generation and tweak inclusion rules #53

Closed jsvine closed 1 year ago

jsvine commented 1 year ago

With the anticipation of adding additional feeds, I figured I'd refactor the code we use to generate them, introducing a new method, generate_feed.

While doing that, I realized it'd be helpful to have slightly more specific rules for whether an entry is included, with the goals of:

palewire commented 1 year ago

As long as we're using the discovered date and not the inspection date I think this is fine. Over the last few weeks I've seen inspections discovered that are dated more than a month ago.

jsvine commented 1 year ago

Yep, sort order is:

And yeahhhh, some of these get posted to APHIS quite a bit after the inspection dates. https://github.com/data-liberation-project/aphis-inspection-reports/issues/36 should help understand some component of that (i.e., how much of this is explained by reports themselves taking a long time to be finalized vs. finalized reports not being uploaded/released).

jsvine commented 1 year ago

(Oh, and yes, the max_age_days limit is based on the discovered date, not the inspection date.)

palewire commented 1 year ago

👍