search

18F / pulse

How the federal .gov domain space is doing at best practices and policies.
Other
94 stars 56 forks source link

use logger module #782

Closed buckley-w-david closed 6 years ago

buckley-w-david commented 6 years ago

This PR replaces the use of print statements throughout the update and processing steps with calls to methods on a logger object.

As requested in #776, here's a paste of what the output of the logger looks like

2018-04-24T10:51:05-0400 - INFO - __main__: Gathering subdomains.
2018-04-24T10:51:05-0400 - INFO - __main__: [gather] Gathering subdomains.
2018-04-24T10:51:05-0400 - INFO - __main__: [cmd] /Users/buckleyd/Projects/domain-scan/gather censys-snapshot,rdns-snapshot,dap,eot2016,other,dotgov --censys-snapshot=https://github.com/GSA/data/raw/master/dotgov-websites/censys-federal-snapshot.csv --rdns-snapshot=https://github.com/GSA/data/raw/master/dotgov-websites/rdns-federal-snapshot.csv --dap=https://analytics.usa.gov/data/live/sites-extended.csv --eot2016=https://github.com/GSA/data/raw/master/end-of-term-archive-csv/eot-2016-seeds.csv --other=https://github.com/GSA/data/raw/master/dotgov-websites/other-websites.csv --dotgov=https://github.com/GSA/data/raw/master/dotgov-domains/current-federal.csv --output=/Users/buckleyd/Projects/pulse/data/./output/subdomains/gather --suffix=.gov,.fed.us --parents=https://github.com/GSA/data/raw/master/dotgov-domains/current-federal.csv --ignore-www --sort --debug
.
.
.

Once a dedicated CLI is put in front of the update and processing modules, that "__main__" will be "data.update" or "data.processing" instead.

konklone commented 6 years ago

:+1: Thank you!