Future option to use 'requests' in place of pycurl where allowed

[svanoort]

This has been raised several times by users as a desired feature, because:

(a) Requests is easier to install (pure-python, installs by pip), none of the native-code libcurl issues especially with virtualenv setups (b) Requests is sometimes more reliable (example: platform performance bugs with DNS lookup and one user reporting poor windows performance of libcurl -- windows isn't supported but still desirable). (c) Requests is more standard except where performance matters.

Previously, this was impossible due to tight library coupling and benchmarks requiring curl. With the refactoring in milestones 1.8 and 2.0 to move run_test/run_benchmark execution logic back into the test/benchmark steps, there will be enough encapsulation to allow it.

Challenges to address:

• PyCurl is generally much faster
  • Possibly something we can address long-term with parallel execution (urgh, or something async)
  • Caveat: there was the start of a dependency chain of steps, but parallel execution requires it + async use
• Which library do we install (and how do we decide?)
  • Requests by default, pycurl if available?
• Automatic selection of pycurl where available vs. requests
  • Use a feature flag REQUEST_LIBRARY in class, import pycurl and set that as the library, otherwise set to requests
• Macros need to be able to fail on load due to missing libraries (pycurl in this case, which is required for benchmarks)