New Feature: Provide deeper `mitmproxy` integration out-of-the-box in Docker

[pirate]

Features mitmproxy provides: https://mitmproxy.org/

• Man-in-the-middle recording,interception,modifying HTTP/HTTPS requests during archiving save_stream_file
  • https://docs.mitmproxy.org/stable/tute-clientreplay/#client-playback-a-30-second-example
• Graphical web UI to inspect/intercept web flows
• Sticky auth + server-side replay, authenticate once in chrome and apply auth cookies automatically on all subsequent requests, even if they're done from other tools like wget/curl/etc.
• Ability to dump entire SSL certificate chain for intercepted requests to cert.pem
  • https://docs.mitmproxy.org/stable/api/mitmproxy/connection.html#Connection.certificate_list
  • https://docs.mitmproxy.org/stable/api/events.html
  • https://lucaslegname.github.io/mitmproxy/2020/04/10/mitmproxy.html

# convert .pem to .crt if needed
openssl x509 -in foo.pem -inform PEM -out foo.crt
# convert .cer to .crt
openssl x509 -inform DER -in foo.cer -out foo.crt
# https://askubuntu.com/questions/73287/how-do-i-install-a-root-certificate/94861#94861

# move mitm CA cert into system store
ln -s /root/.mitmproxy/ca.crt /usr/local/share/ca-certificates/mitmproxy-ca-cert.crt

dpkg-reconfigure ca-certificates
update-ca-certificates
mitmproxy --mode=socks5

# then start chrome pointing to the proxy using socks5
chromium-browser --user-data-dir=/data/personas/Default/chromium --proxy-server='socks5://localhost:8080' --no-sandbox --disable-suid-sandbox --ignore-certificate-errors --ignore-ssl-errors --ignore-certificate-errors-spki-list --allow-running-insecure-content

Alternative option proxy.py: https://github.com/abhinavsingh/proxy.py#run-dashboard

[pirate]

For anyone who finds this via Google, FYI we have: https://github.com/archiveBox/archivebox-proxy