HTTP Engine - Offload Web Requests to Dedicated Process

[TheTechromancer]

TODO:

• [x] Code comments for:
  • [x] request_batch
  • [x] request_custom_batch
• [x] Tests for async cancellation (during request_batch())
• [x] Tests for raise_error=True
• [x] Fix strange race condition causing duplicate IP_ADDRESS (from A module)
• [x] Implement new request_batch() feature:
  • [x] ntlm
  • [x] ajaxpro
  • [x] telerik
  • [x] http_compare
  • [x] paramminer
  • [x] host_header
  • [x] bypass403
  • [x] iis_shortnames
• [x] Fix ctrl+c bug

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 91.33858% with 77 lines in your changes are missing coverage. Please review.

Project coverage is 92%. Comparing base (b16f7d9) to head (bfd62de). Report is 37 commits behind head on bbot-2.0.

:exclamation: Current head bfd62de differs from pull request most recent head 41b8cdc. Consider uploading reports for the commit 41b8cdc to get more accurate results

Files	Patch %	Lines
bbot/core/helpers/web/engine.py	78%	41 Missing :warning:
bbot/core/engine.py	89%	14 Missing :warning:
bbot/core/helpers/web/web.py	93%	5 Missing :warning:
bbot/core/helpers/dns/engine.py	43%	4 Missing :warning:
bbot/modules/iis_shortnames.py	92%	4 Missing :warning:
bbot/scanner/target.py	96%	3 Missing :warning:
bbot/core/core.py	75%	2 Missing :warning:
bbot/modules/ntlm.py	93%	1 Missing :warning:
bbot/modules/output/http.py	50%	1 Missing :warning:
bbot/modules/output/splunk.py	50%	1 Missing :warning:
... and 1 more

Additional details and impacted files

```diff @@ Coverage Diff @@ ## bbot-2.0 #1340 +/- ## ========================================= + Coverage 92% 92% +1% ========================================= Files 334 339 +5 Lines 22451 22901 +450 ========================================= + Hits 20545 21031 +486 + Misses 1906 1870 -36 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.