search

biglocalnews / warn-github-flow

GitHub Action workflow for automating a WARN Act notice ETL pipeline
https://biglocalnews.org/content/tools/layoff-watch.html
Apache License 2.0
5 stars 0 forks source link

Document how to add new scrapers #11

Open stucka opened 11 months ago

stucka commented 11 months ago

Scrapers for newly scraped states need to be manually added to .github/workflows in the etl.yml and extract.yml files.

stucka commented 11 months ago

warn-github-flow also needs to have a branch created with the state's postal abbreviation. Can do that through Github's web interface, code button, easier than anything.

warn-transformer needs sample data in tests/data/raw

scraper and transformer need to both be pushed, versions updated, and the new versions need to be into the pipenv of warn-github flow.