Async::WebSocket::ProtocolError: Failed to negotiate connection: 400 or Protocol::HTTP2::StreamError: Stream closed!

[sasha-id]

Trying to make it work with polygon.io websockets API, and getting these errors, HTTP2/HTTP1: Works fine with faye-websocket-ruby and nio4r-websocket.

Please point out what I am doing wrong.

require 'async'
require 'async/io/stream'
require 'async/http/endpoint'
require 'async/websocket/client'

Async do |task|
  endpoint = Async::HTTP::Endpoint.parse('wss://socket.polygon.io/stocks', alpn_protocols: Async::HTTP::Protocol::HTTP11.names)
  #endpoint = Async::HTTP::Endpoint.parse('wss://socket.polygon.io/stocks)

  Async::WebSocket::Client.connect(endpoint) do |connection|
    puts "Connected..."
    while message = connection.read
      puts "> #{message.inspect}"
    end
  ensure
    puts "close"  
  end
end

endpoint = Async::HTTP::Endpoint.parse('wss://socket.polygon.io/stocks', alpn_protocols: Async::HTTP::Protocol::HTTP11.names)

 4m40s    error: Async::Task [oid=0x6658] [pid=44767] [2020-09-09 19:52:13 -0400]
               |   Async::WebSocket::ProtocolError: Failed to negotiate connection: 400
               |   → /Users/sash/apps/async-websocket/lib/async/websocket/client.rb:100 in `connect'
               |     /Users/sash/apps/async-websocket/lib/async/websocket/client.rb:53 in `block in connect'
               |     /Users/sash/apps/async-websocket/lib/async/websocket/client.rb:44 in `open'
               |     /Users/sash/apps/async-websocket/lib/async/websocket/client.rb:52 in `connect'
               |     (irb):8 in `block in irb_binding'
               |     /Users/sash/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/async-1.26.2/lib/async/task.rb:258 in `block in make_fiber'

endpoint = Async::HTTP::Endpoint.parse('wss://socket.polygon.io/stocks)

  5m3s    error: Async::Task [oid=0x66a8] [pid=44767] [2020-09-09 19:52:36 -0400]
               |   Protocol::HTTP2::StreamError: Stream closed!
               |   → /Users/sash/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/async-http-0.52.4/lib/async/http/protocol/http2/response.rb:109 in `wait'
               |     /Users/sash/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/async-http-0.52.4/lib/async/http/protocol/http2/response.rb:143 in `wait'
               |     /Users/sash/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/async-http-0.52.4/lib/async/http/protocol/http2/client.rb:55 in `call'
               |     /Users/sash/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/protocol-http-0.20.0/lib/protocol/http/request.rb:53 in `call'
               |     /Users/sash/apps/async-websocket/lib/async/websocket/connect_request.rb:97 in `call'
               |     /Users/sash/apps/async-websocket/lib/async/websocket/request.rb:56 in `call'
               |     /Users/sash/apps/async-websocket/lib/async/websocket/client.rb:97 in `connect'
               |     /Users/sash/apps/async-websocket/lib/async/websocket/client.rb:53 in `block in connect'
               |     /Users/sash/apps/async-websocket/lib/async/websocket/client.rb:44 in `open'
               |     /Users/sash/apps/async-websocket/lib/async/websocket/client.rb:52 in `connect'
               |     (irb):22 in `block in irb_binding'
               |     /Users/sash/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/async-1.26.2/lib/async/task.rb:258 in `block in make_fiber'

[ioquatix]

Here is wscat:

Here is async-websocket:

[ioquatix]

It looks like the host header is foobar for some reason.

[ioquatix]

Okay, so this part is now fixed by passing the authority through, but I encountered another issue:

It turns out either my implementation or the server implementation have interpreted the RFC incorrectly.

This seems... odd. The mask has no value when going over SSL/TLS and it's not documented as a "MUST" requirement so enforcing it on the server side is a bit... pedantic. Similar issue discussed here: https://github.com/containous/traefik/issues/4487#issuecomment-467170296

[ioquatix]

Okay, I found more discussion here: https://github.com/gorilla/websocket/issues/560

It turns out section 5.1 clarifies that masking is required in all cases client - server. I did not realise this.

I'm still surprised this is a requirement but I can understand if you have TLS termination -> non-TLS application, you might end up with framing issues.

That being said, this shouldn't be an issue for HTTP/2.

The cost of masking is non-zero, but I guess we have no choice but to enforce it.

[sasha-id]

Thank you for looking into this. Are you going to update your gem to make it work?

[ioquatix]

Yes, I'll provide an update shortly, now that everything is working correctly.

[sasha-id]

Thank you! 🙏

[ioquatix]

Okay, this should be fixed in v0.16.0 and you'll also need to pull the latest protocol-websocket which has some fixes.

[sasha-id]

Thank you so much, it's working! :)

[ioquatix]

Awesome, thanks for the detailed report and using the project.