opencivicdata / pupa

framework for scraping legislative/government data
BSD 3-Clause "New" or "Revised" License
85 stars 42 forks source link

Add pupa clean CLI command #344

Closed antidipyramid closed 1 year ago

antidipyramid commented 1 year ago

Overview

This PR adds a new pupa CLI command: pupa clean:

usage: pupa clean [-h] [--window WINDOW] [--report] [--noinput]

Removes database objects that haven't been seen in recent scrapes

optional arguments: -h, --help show this help message and exit --window WINDOW objects not seen in this many days will be deleted from the database --report only generate a report of what objects this command would delete without making any changes to the database --noinput delete objects without getting user confirmation

Testing Instructions

To test with a live database, you should use a local instance of opencivicdata/scrapers-us-municipal:

coveralls commented 1 year ago

Coverage Status

Coverage: 94.806%. Remained the same when pulling 9f51bbf3f9c5ca8fd3ba048f61c02221ee31c135 on clean-cli into a9c9f80814e92df5ecd0c2525e77d7216ae51949 on master.

hancush commented 1 year ago

Connects https://github.com/Metro-Records/la-metro-councilmatic/issues/900