Improve User Input/Ouput File Transfer

[JoshKarpel]

This PR aims to address #187 and #214 , which are the two sides of the "allow protocol transfers" coin.

• [x] Redo input
• [x] Redo output
• [x] Docs!

The input half is straightforward, and just requires an expansion of the capabilities of the existing htmap.TransferPath. It can now be given a protocol and a location, in addition to the hierarchical path to the actual file/directory. Execute-side path transformation works exactly the same as before . This PR will not address #215 - I'm not convinced we actually need to support that case, given the flexibility of path transformation.

Output file transfer is trickier. As sort-of discussed in #214 , it is possible to remap files that haven't been explicitly named in transfer_output_files. However, we are still stuck with a major problem given how HTMap output file transfer works right now: the file name and destination must be named in transfer_output_remaps before the job starts running. For us, that means the user needs to declare the output file name and destination as part of creating the map. Thoughts ongoing on how to accomplish this cleanly.

Resolves #187 Resolves #214

[codecov[bot]]

Codecov Report

Merging #216 into v0.6.0 will increase coverage by 4%. The diff coverage is 84%.

@@           Coverage Diff           @@
##           v0.6.0   #216     +/-   ##
=======================================
+ Coverage      79%    84%     +4%     
=======================================
  Files          19     77     +58     
  Lines        2007   3633   +1626     
  Branches      304    391     +87     
=======================================
+ Hits         1591   3053   +1462     
- Misses        356    496    +140     
- Partials       60     84     +24     

Impacted Files	Coverage Δ
htmap/checkpointing.py	26% <0%> (ø)
htmap/tags.py	79% <ø> (ø)
...ation/user_file_transfer/test_fixed_input_files.py	91% <ø> (ø)
...ion/user_file_transfer/test_url_output_transfer.py	36% <36%> (ø)
htmap/condor.py	55% <55%> (ø)
htmap/options.py	81% <66%> (-4%)	:arrow_down:
htmap/utils.py	74% <66%> (-2%)	:arrow_down:
htmap/management.py	47% <71%> (+1%)	:arrow_up:
htmap/maps.py	78% <78%> (+<1%)	:arrow_up:
...tegration/user_file_transfer/test_output_remaps.py	82% <82%> (ø)
... and 78 more

[JoshKarpel]

https://github.com/htcondor/htmap/pull/216/commits/8d91538be50d2b2117864b33a997e0f49c58efb9 enables a worse form of URL output files, where users must specify their remaps ahead-of-time via MapOptions.

This PR will also contain a turned-off implementation of late-binding URL output transfer via a custom plugin. It's not turned on for two reasons:

• It doesn't work with s3 file transfer, because s3 file transfer is done by pre-signed URLs generated by the shadow. We have no easy way to do this in the plugin right now. We could use HTChirp to fetch the user's s3 credentials, but that connection may not be secure, so sending tokens over it is bad.
• It requires Python 3 and the Python 3 HTCondor bindings to be installed on bare metal (the plugin cannot run inside the container). This is easy to guarantee in testing, but hard/impossible to guarantee on, say, the OSG.

I think its day will still come, but that day is not today...