Support non-HTTP/Websocket protocols (upgrade/CONNECT/Socks)

[shapovalovts]

After upgrade to gun-1.0.0-pre.5 our WS upgrade code was broken. Instead of gun_ws_upgrade now gun_http.erl sends undocumented (in websocket.asciidoc) gun_inform message and nothing more.

Some investigation has shown that if the following case was commented out in gun_http.erl then the WS upgrade worked fine again:

207     {_, _} when Status >= 100, Status =< 199 ->                                                                                      
208       ReplyTo ! {gun_inform, self(), stream_ref(StreamRef), Status, Headers},                                                        
209       handle(Rest2, State); 

What was the idea behind those 3 lines added 5 month ago?

[essen]

The 1xx responses are now supported. Are you saying that you do not receive the gun_ws_upgrade at all anymore, or that you receive this before receiving the gun_ws_upgrade?

[shapovalovts]

Yes, after ignored gun_inform message nothing else come anymore to my client process, so that new case in gun_http seems somehow stops the WS-upgrade continuation. Downgrading to pre4 solves our issue.

[essen]

Please show me how you initiate the Websocket connection, what headers get sent and received and any other relevant information. The Websocket test suite is working so it must be a very specific issue.

[shapovalovts]

Thanks to looking into the issue. I have spent some time before figured out that the issue is actually is in different place (web sockets look good). Let me explain and maybe it makes sense to create a new issue, because the header of this one is not relevant any more.

Our erlang code communicates to docker daemon. Using gun we attach to a docker container stdin/out streams. Docker provides 2 ways to attach to the contianer: using POST and WS, they have some differences and for now we use the both in different scenarions (thus I confused WS and POST output in the first place).

I created a test escript to demonstrate what really happens:

#!/usr/bin/env escript
-define(DOCKER_PORT, 6000).
-define(DOCKER_HOST, "127.0.0.1").

run_test() ->
  Opts = #{retry_timeout => 5000,
           http_opts =>
             #{keepalive => 5000}
          },
  {ok, Pid} = gun:open(?DOCKER_HOST, ?DOCKER_PORT, Opts),
  {ok, http} = gun:await_up(Pid),
  monitor(process, Pid),
  ID = os:getenv("CONTAINER"),
  Params = "?logs=1&stream=1&stderr=1&stdout=1&stdin=0",
  Path = "/containers/" ++ ID ++ "/attach" ++ Params,
  Hdr = [{<<"Content-Type">>, "application/vnd.docker.raw-stream"},
          {<<"Upgrade">>, "tcp"},
          {<<"Connection">>, "Upgrade"}],
  gun:post(Pid, Path, Hdr, []),
  recv_loop().

recv_loop() ->
  receive
    Msg ->
      io:format("~p~n", [Msg])
  end,
  recv_loop().

main(_) ->
  true = code:add_path(os:getenv("GUN_DIR")),
  true = code:add_path(os:getenv("COW_DIR")),
  true = code:add_path(os:getenv("RANCH_DIR")),
  io:format("~p~n", [application:ensure_all_started(gun)]),
  run_test().

In order to test the scenario one can run a simple docker container (in a separate terminal, it is interractive):

docker run -ti ubuntu cat

If you then run the escript in a different terminal, then you should get a gun_data messages every time when you put some characters and press Enter into the docker container terminal. For example:

$ docker run -ti ubuntu cat
hello ENTER
hello

And the escript with gun version pre4 will show (after required variables are exported):

$ ~/gun-docker-test.escript 
{ok,[crypto,asn1,public_key,ssl,cowlib,ranch,gun]}
{gun_response,<0.84.0>,#Ref<0.1379562992.1504182273.134338>,nofin,101,
              [{<<"content-type">>,<<"application/vnd.docker.raw-stream">>},
               {<<"connection">>,<<"Upgrade">>},
               {<<"upgrade">>,<<"tcp">>}]}
{gun_data,<0.84.0>,#Ref<0.1379562992.1504182273.134338>,nofin,<<"d">>}
{gun_data,<0.84.0>,#Ref<0.1379562992.1504182273.134338>,nofin,<<"\b \b">>}
{gun_data,<0.84.0>,#Ref<0.1379562992.1504182273.134338>,nofin,<<"h">>}
{gun_data,<0.84.0>,#Ref<0.1379562992.1504182273.134338>,nofin,<<"e">>}
{gun_data,<0.84.0>,#Ref<0.1379562992.1504182273.134338>,nofin,<<"l">>}
{gun_data,<0.84.0>,#Ref<0.1379562992.1504182273.134338>,nofin,<<"l">>}
{gun_data,<0.84.0>,#Ref<0.1379562992.1504182273.134338>,nofin,<<"o">>}
{gun_data,<0.84.0>,#Ref<0.1379562992.1504182273.134338>,nofin,<<"\b \b">>}
{gun_data,<0.84.0>,#Ref<0.1379562992.1504182273.134338>,nofin,<<"\b \b">>}
{gun_data,<0.84.0>,#Ref<0.1379562992.1504182273.134338>,nofin,<<"\b \b">>}
{gun_data,<0.84.0>,#Ref<0.1379562992.1504182273.134338>,nofin,<<"\b \b">>}
{gun_data,<0.84.0>,#Ref<0.1379562992.1504182273.134338>,nofin,<<"\b \b">>}
{gun_data,<0.84.0>,#Ref<0.1379562992.1504182273.134338>,nofin,<<"t">>}
{gun_data,<0.84.0>,#Ref<0.1379562992.1504182273.134338>,nofin,<<"e">>}
{gun_data,<0.84.0>,#Ref<0.1379562992.1504182273.134338>,nofin,<<"s">>}
{gun_data,<0.84.0>,#Ref<0.1379562992.1504182273.134338>,nofin,<<"t">>}
{gun_data,<0.84.0>,#Ref<0.1379562992.1504182273.134338>,nofin,
          <<"\r\ntest\r\n">>}

But when you use pre5, then it looks like that:

$ ~/gun-docker-test.escript
{ok,[crypto,asn1,public_key,ssl,cowlib,ranch,gun]}
{gun_inform,<0.83.0>,#Ref<0.2398647491.4184080385.5630>, 101,
              [{<<"content-type">>,<<"application/vnd.docker.raw-stream">>},
               {<<"connection">>,<<"Upgrade">>},
               {<<"upgrade">>,<<"tcp">>}]}

No one message comes after gun_inform message. After awhile this message comes though: {gun_error,<0.83.0>,#Ref<0.2398647491.4184080385.5630>, {closed,"The connection was lost."}}

[essen]

Because it doesn't support non-Websocket upgrades. It's waiting for the final response. OK so we need to handle non-Websocket Upgrade/101. Note that both the current and previous states of affairs are incorrect, though you could do something with the previous one.

Not sure what the interface should look like. Could just do data packets similar to how it was working previously and let the user deal with it I guess, or could have some kind of handler that receives data and forwards events as they get rebuilt rather than having to deal with raw data directly (basically democratizing the interface that already exists for Websocket). I'd lean for the latter though that's more work, so perhaps first one for v1 as a quick fix, and second one for v2.

[essen]

The plan is to have a special state and internal protocol for non-Websocket protocol upgrades as well as CONNECT to non-http/socks servers. Let's call it raw. This requires adding a function for performing HTTP/1.1 upgrades to non-Websocket servers, as well as adding the state/protocol for this and a function to send data. To receive data we can just use the flow feature to control how much we want to receive.

[essen]

This should now work as of a3c2edbb8c807717e2f10520c6cf1e77a62eab2e. It's still very rough but will be improved and documented before the 2.0 release. The raw "protocol" will be allowed everywhere where the HTTP protocol is allowed, and currently this means direct connections, after Socks/CONNECT and after non-Websocket HTTP/1.1 Upgrades. We will most likely want flow control for this as well.

Please experiment and provide feedback in this ticket or open new ticket with issues you encounter. Closing, thanks!