Python cleanup

[LRWeber]

Full repo cleanup via Black and Flake8. Additional typo fixes.

Note: There are still some Flake8 errors in the older code in the scripts directory, specifically regarding unused and unassigned variables. They should be safe to ignore as those scripts are unused and kept as reference material.

[leebrian]

Would you link to the flake8 errors? I just see these, from commit https://github.com/LLNL/scraper/pull/47/commits/63ce5b894fc28a9be8f3933e5ead92bb62e61a47 (starting on line 323

scraper/util.py:53:79: E231 missing whitespace after ','
scraper/util.py:54:83: E231 missing whitespace after ','
scraper/util.py:55:83: E231 missing whitespace after ','
scraper/code_gov/__init__.py:40:61: E231 missing whitespace after ','
scraper/github/queryManager.py:74:17: W503 line break before binary operator
scraper/github/queryManager.py:75:17: W503 line break before binary operator
scraper/github/queryManager.py:76:17: W503 line break before binary operator

[leebrian]

I get a clean result from "flake8 --ignore=E231,E501,W503 scraper/" on your branch...

[LRWeber]

Those errors were introduced by Black's formatting choices and should be resolved now that I added the appropriate ignore options.

The additional problems show up when I run flake8 by hand, but aren't caught by Travis. ~I'm guessing it's set to not reject the less critical FXXX error codes.~

.\scripts\get_stargazers.py:20:40: F821 undefined name 'file_path'
.\scripts\get_stargazers.py:51:32: F821 undefined name 'raw_input'
.\scripts\get_stargazers.py:92:20: F821 undefined name 'raw_input'
.\scripts\get_stargazers.py:102:38: F821 undefined name 'raw_input'
.\scripts\get_stargazers.py:161:21: F841 local variable 'count' is assigned to but never used
.\scripts\get_traffic.py:34:40: F821 undefined name 'file_path'
.\scripts\get_traffic.py:89:32: F821 undefined name 'raw_input'
.\scripts\get_traffic.py:130:20: F821 undefined name 'raw_input'
.\scripts\get_traffic.py:140:38: F821 undefined name 'raw_input'
.\scripts\get_traffic.py:220:9: F841 local variable 'url_paths' is assigned to but never used
.\scripts\get_traffic.py:337:9: F841 local variable 'previous_dates' is assigned to but never used
.\scripts\get_users_emails.py:21:13: F841 local variable 'count_members' is assigned to but never used
.\scripts\get_users_emails.py:46:32: F821 undefined name 'raw_input'
.\scripts\get_users_emails.py:87:20: F821 undefined name 'raw_input'
.\scripts\get_users_emails.py:96:33: F821 undefined name 'raw_input'
.\scripts\get_year_commits.py:58:32: F821 undefined name 'raw_input'
.\scripts\get_year_commits.py:99:20: F821 undefined name 'raw_input'
.\scripts\get_year_commits.py:108:33: F821 undefined name 'raw_input'
.\scripts\get_year_commits.py:134:17: F841 local variable 'total' is assigned to but never used
.\scripts\github_stats.py:139:32: F821 undefined name 'raw_input'
.\scripts\github_stats.py:180:20: F821 undefined name 'raw_input'
.\scripts\github_stats.py:189:33: F821 undefined name 'raw_input'
.\scripts\github_stats.py:327:13: F841 local variable 'count' is assigned to but never used
.\scripts\github_stats.py:331:41: F821 undefined name 'basestring'
.\scripts\github_stats.py:338:25: F841 local variable 'size' is assigned to but never used
.\scripts\stars.py:35:11: F821 undefined name 'json_data'
.\scripts\stars.py:36:23: F821 undefined name 'json_data'
.\scripts\stars.py:38:9: F841 local variable 'json_data' is assigned to but never used

For the record, I'm working with flake8 3.7.9

> flake8 --version
3.7.9 (mccabe: 0.6.1, pycodestyle: 2.5.0, pyflakes: 2.1.1) CPython 3.8.0 on Windows

[LRWeber]

I see now; Travis is specifically running on the scraper/ sub directory, so it isn't reviewing anything in scripts/ at all.

Those scripts are old and I think mostly just there for reference at this point anyway. We should be able to safely ignore them for now, but they may be worth revisiting (or deleting) later.

[leebrian]

Ok cool, that’s why it was showing clean for me too, I was running just on scraper.

In that case, I think the PR will make it easier for maintenance and future work.