Multiple fixes, feature adds, cleanup. Also partially completes issue 44…

[sedennial]

… #44

Fixes:

• Renamed upload function to do_upload as we use the option 'upload' as a boolean elsewhere. Confusing.
• Fixed a missing default for 'upload' in multiparser class that wasn't allowing 'upload' to be used as a config file option.

Cleanup/Housekeeping:

• Changed some errors handling from 'raise error' to simply print error message and exit with a exit code of one (1). Some errors such as invalid extension or file not found don't need the traceback they just need to output notification to the user and set the shell exit code as they are user errors, not code errors.
• Moved some variables to upload since they are only needed there.
• Moved the story builder inside a conditional to add ability to skip story generation via command line switch (--nostory), or individual story option. Will allow you to temporarily turn off a particular story without removing it from your config or leave your workflow in place without actually deleting your tasks/job. Also handy for testing upload functionality from the command line.

New features:

• Added --nostory option. Allows you to skip all the story generation and process an upload only. In this case the 'output' file is used as the file to upload directly.
• Added '--noreplace' and '--noupload' as command line only options to allow overriding config file settings (for testing (or other uses ?)) without having to constantly edit config files.
• Added 'upload' specified in config or global
• Added 'output' can be specified in config, global, or command line.
• Added sanity checking if you provide more than one 'output'.
• Created default config files
• Added replace to default options as 'False'
• Added cleanup option (delete target goosepaper after upload success)
• Added formatter class to multiparser to allow newlines (optionally) used in help for command line options. Makes for cleaner reading on some help text (IMO). Mostly mine since my default setting is --verbose --verbose. :)
• Added a per story 'skip' option (similar to nostory global option but per story source) which will allow you to skip processing a particular story provider without having to remove your entire story config. Useful if the source is having issues but you don't want to remove your entire entry from that config.
• Added an entry to examples to show use of 'skip'
• Added checks to make sure that either at least one default config file exists or you specified one on the command line.
• Added global config preference files in order of precedence. Command line options superseded all config file settings.
  • ~/.goosepaper.json (HOME environment from os)
  • ./goosepaper.json (current working directory when goosepaper called)
  • --config (specified as argument)
• Added '--showconfig' command line switch to allow both user and dev debugging of config file and command line options since there are four ways to pass settings and options. Shows each file in order parsed as well as the final config
• Added goosepaper.json containing all the (current) operational variables set to operate with the least impact to any existing workflow process (i.e. no change to current behavior as of 0.3.1).

[codecov[bot]]

Codecov Report

Merging #48 (6616f6d) into master (50dc649) will decrease coverage by 4.25%. The diff coverage is 2.52%.

:exclamation: Current head 6616f6d differs from pull request most recent head 1ad5383. Consider uploading reports for the commit 1ad5383 to get more accurate results

@@            Coverage Diff             @@
##           master      #48      +/-   ##
==========================================
- Coverage   39.09%   34.83%   -4.26%     
==========================================
  Files          19       19              
  Lines         550      620      +70     
==========================================
+ Hits          215      216       +1     
- Misses        335      404      +69     

Impacted Files	Coverage Δ
goosepaper/__main__.py	0.00% <0.00%> (ø)
goosepaper/auth.py	0.00% <ø> (ø)
goosepaper/multiparser.py	0.00% <0.00%> (ø)
goosepaper/rss.py	37.50% <ø> (ø)
goosepaper/upload.py	0.00% <0.00%> (ø)
setup.py	0.00% <ø> (ø)
goosepaper/util.py	81.81% <50.00%> (-1.21%)	:arrow_down:
goosepaper/goosepaper.py	41.48% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 236978e...1ad5383. Read the comment docs.

[sedennial]

I do hesitate a little bit to have upload-only code in here (i.e. a code path that JUST uploads and has nothing to do with paper generation). I worry that we're starting to mix code purposes a little, and maybe upload-specific workflows should use something like rmapy directly.

I kinda thought about that too, but one reason I did it was if you have a lot of goosepapers (I have five separate ones which take a while to run) the user could have them created, and then have a different upload process entirely if they wanted. Which is how I was using it initially. While right now the users probably have the ability to use rmapy to do standalone uploads, it would be nice for less programmatic users to be able to use GP to handle that too. Right now I'm also using pandoc to produce some daily outputs from non-RSS sites (https://isc.sans.org) for example and I'm using GP to handle the upload.

Also in line with the possibility of sending update, documentation, errors, etc. to the RM, this gives GP the ability to compile multiple logs from multiple runs, convert them, and then upload them.

All of that said I'm not going to dig in my heels if you want to take it out. I'll just keep it in my private branch. :)

[j6k4m8]

@sedennial — I had some stylistic nitpicks which is why it's taken me so long to get back to this — and then I realized that if I really was going to be such a stickler, there were plenty of places to fix my own code first :) Very sorry for the lag here.

My main priority is making sure that this is backwards-compatible with prior code (i.e., someone can update their goosepaper install and keep using their scripts as-is). So far I think this is true, but insight welcome! Otherwise, I think this is ready to merge, right?