opencivicdata / pupa

framework for scraping legislative/government data
BSD 3-Clause "New" or "Revised" License
85 stars 42 forks source link

Add failsafe to pupa clean command #346

Closed hancush closed 1 year ago

hancush commented 1 year ago

It's probably not desirable for the clean command to wipe out a significant portion of the database. Add some logic to prompt for confirmation if clean will remove more than a fixed number (or some percentage of) data from a given table.

hancush commented 1 year ago

Connects https://github.com/Metro-Records/la-metro-councilmatic/issues/900

antidipyramid commented 1 year ago

@hancush The pupa clean does have confirmation logic already:

            if not args.noinput:
                num_to_delete = len(list(self.get_stale_objects(args.window)))
                print(
                    f"This will permanently delete"
                    f" {num_to_delete}"
                    " objects from your database"
                    f" that have not been scraped within the last {args.window}"
                    " days. Are you sure? (Y/N)"
                )
                resp = input()
                if resp != "Y":
                    sys.exit()

Do we want to have two prompts to confirm?

hancush commented 1 year ago

@antidipyramid Ah, that should suffice. Thanks!