zschira commented 2 years ago

This PR adds a script for pulling XBRL archives from the RSS feed provided by FERC

Filings are downloaded and written to a zipfile
There are options to specify the filing number and year
This script will also archive metadata, which indicates the date each filing was published, so if multiple filings from the same filer and period exist PUDL can distinguish which is the most recent

codecov[bot] commented 2 years ago

Codecov Report

Merging #28 (101cf93) into main (fed8b9f) will increase coverage by 0.6%. The diff coverage is 69.8%.

@@           Coverage Diff           @@
##            main     #28     +/-   ##
=======================================
+ Coverage   64.7%   65.3%   +0.6%     
=======================================
  Files         15      16      +1     
  Lines        542     615     +73     
=======================================
+ Hits         351     402     +51     
- Misses       191     213     +22

Impacted Files	Coverage Δ
src/pudl_scrapers/bin/ferc_xbrl.py	`69.8% <69.8%> (ø)`

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

zschira commented 2 years ago

I'm going to close this PR. The XBRL scraper has been largely rewritten to support the new segmented RSS feed. There will also be more changes coming in the future with scripts coordinating XBRL and DBF scraping.

catalyst-cooperative / pudl-scrapers

Xbrl integration #28

Codecov Report