Closed jdangerx closed 8 months ago
This causes data to get dropped when we are reading data from the FercXbrlSqliteExtractor
, since it uses filing_name
to join the data table with an ID table. If the ID table and the data table choose different filings from the same report date, then the data won't get found in that join.
For FERC forms 1, 2, 6, and 60, we expect each filing to include a
ReportDate
fact, and for FERC 714 we expect each filing to include aCertifyingOfficialDate
fact.We use these in the
ferc-xbrl-extractor
to order the filings by recency, so we can merge all the filings and use the most recent data we have for any given fact.However, these only have day-level granularity, and often filings share the same date fact but are published at different times. We can tell because we can see the publish time with high granularity from the RSS feed metadata, which we already track.
To avoid ambiguity, we should use that RSS feed metadata instead of the report's self-reported date to determine which report should take precedence.