Clone from and push to any place reachable by any git-annex special remote

[mih]

This PR contains:

• [x] a base class to implement external git-annex backends (derived from annexremote)
• [x] a rough implementation of a XDLCLN annex backend, based on ideas from https://github.com/datalad/datalad/issues/5221#issuecomment-902276109 git-annex external backend and an annex-based git-remote helper
• [x] a git-remote-datalad-annex git remote helper implementation that uses an ephemeral/throwaway repository to interface git-annex special remotes to transfer two special XDLCLN keys with the refs and archive of an entire repository (this code essentially replaces the datalad-osf functionality with a generic implementation)
• [x] Test for clone repo from any http(s) URL via the web-type special remote
• [x] Test for clone repo from any http(s) URL via the web-type special remote in export mode
• [x] a git-remote-datalad-annex that can also work with exporttree=yes special remotes
• [x] Documentation for XDLRA
• [x] Test git-remote-datalad-annex on windows (blocked by https://github.com/datalad/datalad/issues/6335, worked around by using the -dev version on windows), also requires a replacement for the use of file:// URLs (at least a POSIX notation conversion)
• [x] Verify git-remote-datalad-annex function with non-UTF encoded commits (try push --all in main datalad repo)
• [x] make LZMA-compression optional (can be counter-productive performance-wise when the network is fast
• [x] Test behavior when import succeeds for git, but the last upload step fails
• [x] Test annex-mode and export-mode in analog fashion
• [x] Test successful and complete re-push after remote deposit got removed unknowingly.
• [x] Manually test push/clone roundtrip on WEBDAV (export, and regular annex)
• [x] RF constants into a single place
• [x] Test cross-platform data change (from created-on-windows via WEBDAV to linux, and back)
• [x] Test push/fetch of a non-heads reference (and also update-necessity detection)
• [x] Test utilization as submodule URLs
• [x] Try rclone-based special remotes (tested dropbox via rclone manually)
• [ ] Figure out why git-annex on windows fails to force-drop from a local annex (with permission denied, key file is read-only), but only in one test, and not in another. https://git-annex.branchable.com/bugs/Fails_to_drop_key_on_windows___40__Access_denied__41__/?updated
• [x] datalad credential integration
• [x] test for datalad credential integration

Wanna try it?

# to push any repo to a directory special remote
git remote add dla 'datalad-annex::file:///tmp/myds?type=directory&directory={path}&encryption=none'
# and then `git push`

# to clone it from there
git clone 'datalad-annex::file:///tmp/myds?type=directory&directory={path}&encryption=none'

# also try adding `&exporttree=yes` to the list of parameters in the URL, and take a look at
# `/tmp/myds`

[codecov[bot]]

Codecov Report

Merging #9 (9fb8f56) into master (0cc9b2d) will decrease coverage by 25.05%. The diff coverage is 35.00%.

@@             Coverage Diff             @@
##           master       #9       +/-   ##
===========================================
- Coverage   81.68%   56.62%   -25.06%     
===========================================
  Files          11       15        +4     
  Lines         535     1155      +620     
===========================================
+ Hits          437      654      +217     
- Misses         98      501      +403     

Impacted Files	Coverage Δ
datalad_mihextras/_version.py	100.00% <ø> (ø)
datalad_mihextras/backend/base.py	0.00% <0.00%> (ø)
datalad_mihextras/backend/xdlra.py	0.00% <0.00%> (ø)
datalad_mihextras/gitremote/datalad_annex.py	20.71% <20.71%> (ø)
...ad_mihextras/gitremote/tests/test_datalad_annex.py	97.45% <97.45%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 0cc9b2d...9fb8f56. Read the comment docs.

[codeclimate[bot]]

Code Climate has analyzed commit 9fb8f563 and detected 11 issues on this pull request.

Here's the issue category breakdown:

Category	Count
Complexity	8
Security	3

View more on Code Climate.