fedup: Parallelize package downloads with "async"

[cemeyer]

Normal "yum update" does this -- there's no reason fedup should not, when it deals with even larger sets of packages. This could help the "prepare to upgrade" stage quite a bit.

Testing without this patch shows single connection at a time behavior; with patch, we fetch from multiple mirrors. (iftop.)

Note that the UI is a little bit broken -- we lose the progress bar. Somehow yum manages to keep it in the face of async downloads, so I suspect it's possible to fix. But I'd rather have parallel downloads than a progress bar any day ;-).

[cemeyer]

Note, for origin/legal purposes: This is my original (though trivial) work, and you can do whatever you want with it under the MIT or GPLv2 license (or probably anything else, I think MIT should cover most everything).

[cemeyer]

The second patch makes fedup resilient to transient server errors and/or bad mirrors. With both patches my average network speed (as measured by iftop) is roughly line speed, despite fetching a ton of tiny packages (F18->F19 texlive pkgs).

Like the first patch, this is my original work, released under the same liberal licensing blah blah.

[tomspur]

fedup/download.py is under GPLv2+. Why don't you just use that? It is not so easy to relicence from GPL to MIT anyway...

[cemeyer]

@tomspur What's your question? If my changes are released under MIT you're perfectly welcome to incorporate them in a GPLv2+ source base like fedup. To be clear: my changes being under MIT does not in any way cause download.py to magically switch licenses. In a post-merged world, download.py is still GPLv2+.

[tomspur]

I doubt that you are allowed to choose another licence as the original one from fedup/download.py, when doing your changes in the first place.

See https://github.com/wgwoods/fedup/blob/master/fedup/download.py#L8 here for the "or later versions".

[cemeyer]

@tomspur My changes are my original work. They're pretty useless on their own, but on their own I see no reason why they can't be MIT licensed.

Once you apply them to the GPLv2+ work that is fedup, of course, the collective work is GPLv2+.

And yes, they're pretty pointless without being applied on top of fedup...

Still, is there an actual issue here? Perhaps something technically wrong? One can bikeshed about licenses all day, but I'd rather just make fedup better...

[wgwoods]

I've added async downloading (with the proper progress bar!) in 7dcbade. The retry handling will be added once I figure out how yum handles that internally.

[cemeyer]

Awesome. Thanks @wgwoods!

[wgwoods]

Okay! commit 2d2c958 should fix the failover/retry problems - we were getting in our own way. Brief testing shows that failures get logged and then retried later, as expected. Hooray!