Implement proxy middleware

[insiderq]

Is it possible to build such a middleware? I heard that issue many times in our local community

[secretworry]

@insiderq The short answer is yes, but will make the middleware be adapter-specified.

We haven't built anything related to adapting proxy, but it won't be hard to use a proxy if the client supports configuring proxy using opts. Just to create a middleware, add proxy-related opts in the Conn.opts. Those opts will be passed directly to the client as its opts arg.

For example, for ibrowser, you could just add the following to the opts

[proxy_user: "XXXXX", proxy_password: "XXXXX", proxy_host: "proxy", proxy_port: 8080]

and those opts will be passed to :ibrowser.send_req as the fifth arg.

How to use proxy for a specified client could be found in the list following

• ibrowser - "A GET using a proxy"
• httpc - "request(Method, Request, HTTPOptions, Options)"

@zhongwencool We may need to support adapting proxy in the next release by adding proxy-related configuration in the Conn

[insiderq]

Thank you for the extended and fast answer!

[secretworry]

@insiderq Thanks for using Maxwell :)

[zhongwencool]

https://github.com/benoitc/hackney/blob/9caf8c612d2408498b95da550755c4ab71e48ac2/src/hackney.erl#L242 @secretworry hackney can. @insiderq keep this issue open, I will implement proxy middleware this week.

[secretworry]

@zhongwencool aha~ I though the hackey should use hackney_http_proxy:connect_proxy, but it seems the call is for reusing a connection. @insiderq my mistake for declaring the hackney isn't available for using proxy :P

[zhongwencool]

@secretworry I find it's very hard to implement this middleware gracefully. https://github.com/zhongwencool/maxwell/pull/41 I'm block by httpc, httpc can't setting proxy(host, port) without httpc:set_options/2 I don't know where to call set_options/2 ?

---

Other solution: just setting proxy form Maxwell.Middleware.Opts

1. hackney: https://github.com/benoitc/hackney/blob/f69567eb28d6cbf2fdf820100055fca5e6c66938/doc/README.md#proxy-a-connection

##http proxy
middleware Maxwell.Middleware.Opts, [proxy: proxy_url}, proxy_auth: {user, password}]
## socket5 proxy
middleware Maxwell.Middleware.Opts, [proxy: {:socks5, proxy_host, proxy_port},
          socks5_user: username, socks5_pass: password]

2. ibrowse: https://github.com/cmullaparthi/ibrowse/blob/303bc1fc9d074af4700e0315d5e19d4865675143/src/ibrowse.erl#L27

   middleware Maxwell.Middleware.Opts, [proxy_host: host_char_list, proxy_port: port_integer, 
   proxy_user: user_char_list,  proxy_password: password_char_list]

3. httpc http://erlang.org/doc/man/httpc.html#set_options-1 User call httpc:set_options/2 to setting proxy(host port) in a proper place.

   middleware Maxwell.Middleware.Opts,  [proxy_auth: {user_char_list, passwd_char_list}]

[secretworry]

@zhongwencool I think the proxy action should be an adapter-specified action, so the middleware should just put in the generalized proxy options, and let the adapters to handle setting up the proxy. We can not rely on the adapter's implementation for proxy using opts.

[zhongwencool]

You mean, add proxy field in conn instead put proxy into options:

%Conn{proxy = keyword_list }

and add Conn.set_proxy/2 function.

if we want to add a cookie middleware. we need add cookie filed in conn

%Conn{cookie}

I'm afraid that conn will get bigger and bigger after a period of time.

---

Beside, user can't add new field in conn when they want to write their own middleware. How to solve this problem?

[bitwalker]

I Think #35 solves the issue of users not being able to add new fields to conn.

[imranismail]

I'll work on that this weekend 😄