EOFError: encountered eof while reading data

[danielmconrad]

Under a normal amount of load, we've begun to see the below error surface. Once we see this error, all subsequent requests will fail, including our health check, which is as simple as ActiveGraph::Base.read_transaction { |tx| tx.run('MATCH (n) RETURN n LIMIT 1') }.

Database: Neo4j 3.5, 4.4

Gems:

gem 'activegraph', '11.1.0.alpha.4'
gem 'neo4j-ruby-driver', '4.4.0.beta.1'

Error:

EOFError: encountered eof while reading data

File /usr/local/bundle/gems/async-io-1.34.0/lib/async/io/stream.rb:114 in read_exactly
File /usr/local/bundle/gems/neo4j-ruby-driver-4.4.0.beta.1/ruby/neo4j/driver/internal/packstream/pack_input.rb:26 in read_int
File /usr/local/bundle/gems/neo4j-ruby-driver-4.4.0.beta.1/ruby/neo4j/driver/internal/async/connection/handshake_handler.rb:42 in decode
File /usr/local/bundle/gems/neo4j-ruby-driver-4.4.0.beta.1/ruby/neo4j/driver/internal/async/pool/channel.rb:16 in initialize
File /usr/local/bundle/gems/neo4j-ruby-driver-4.4.0.beta.1/ruby/neo4j/driver/internal/async/pool/connection_pool_impl.rb:112 in new
File /usr/local/bundle/gems/neo4j-ruby-driver-4.4.0.beta.1/ruby/neo4j/driver/internal/async/pool/connection_pool_impl.rb:112 in block in new_pool
File /usr/local/bundle/gems/connection_pool-2.2.5/lib/connection_pool/timed_stack.rb:169 in try_create
File /usr/local/bundle/gems/connection_pool-2.2.5/lib/connection_pool/timed_stack.rb:73 in block (2 levels) in pop
File /usr/local/bundle/gems/connection_pool-2.2.5/lib/connection_pool/timed_stack.rb:69 in loop
File /usr/local/bundle/gems/connection_pool-2.2.5/lib/connection_pool/timed_stack.rb:69 in block in pop
File /usr/local/bundle/gems/connection_pool-2.2.5/lib/connection_pool/timed_stack.rb:68 in synchronize
File /usr/local/bundle/gems/connection_pool-2.2.5/lib/connection_pool/timed_stack.rb:68 in pop
File /usr/local/bundle/gems/neo4j-ruby-driver-4.4.0.beta.1/ruby/neo4j/driver/internal/async/pool/controller.rb:12 in acquire
File /usr/local/bundle/gems/neo4j-ruby-driver-4.4.0.beta.1/ruby/neo4j/driver/internal/async/pool/connection_pool_impl.rb:22 in acquire
File /usr/local/bundle/gems/neo4j-ruby-driver-4.4.0.beta.1/ruby/neo4j/driver/internal/cluster/rediscovery_impl.rb:100 in lookup_on_router
File /usr/local/bundle/gems/neo4j-ruby-driver-4.4.0.beta.1/ruby/neo4j/driver/internal/cluster/rediscovery_impl.rb:85 in block in lookup_on_known_routers
File /usr/local/bundle/gems/neo4j-ruby-driver-4.4.0.beta.1/ruby/neo4j/driver/internal/cluster/rediscovery_impl.rb:86 in each
File /usr/local/bundle/gems/neo4j-ruby-driver-4.4.0.beta.1/ruby/neo4j/driver/internal/cluster/rediscovery_impl.rb:86 in each
File /usr/local/bundle/gems/neo4j-ruby-driver-4.4.0.beta.1/ruby/neo4j/driver/internal/cluster/rediscovery_impl.rb:86 in each
File /usr/local/bundle/gems/neo4j-ruby-driver-4.4.0.beta.1/ruby/neo4j/driver/internal/cluster/rediscovery_impl.rb:86 in each
File /usr/local/bundle/gems/neo4j-ruby-driver-4.4.0.beta.1/ruby/neo4j/driver/internal/cluster/rediscovery_impl.rb:86 in first
File /usr/local/bundle/gems/neo4j-ruby-driver-4.4.0.beta.1/ruby/neo4j/driver/internal/cluster/rediscovery_impl.rb:86 in lookup_on_known_routers
File /usr/local/bundle/gems/neo4j-ruby-driver-4.4.0.beta.1/ruby/neo4j/driver/internal/cluster/rediscovery_impl.rb:74 in lookup_on_known_routers_then_on_initial_router
File /usr/local/bundle/gems/neo4j-ruby-driver-4.4.0.beta.1/ruby/neo4j/driver/internal/cluster/rediscovery_impl.rb:68 in lookup
File /usr/local/bundle/gems/neo4j-ruby-driver-4.4.0.beta.1/ruby/neo4j/driver/internal/cluster/rediscovery_impl.rb:33 in lookup_cluster_composition
File /usr/local/bundle/gems/neo4j-ruby-driver-4.4.0.beta.1/ruby/neo4j/driver/internal/cluster/routing_table_handler_impl.rb:36 in block in ensure_routing_table
File /usr/local/bundle/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/atomic/reentrant_read_write_lock.rb:147 in with_write_lock
File /usr/local/bundle/gems/neo4j-ruby-driver-4.4.0.beta.1/ruby/neo4j/driver/internal/cluster/routing_table_handler_impl.rb:30 in ensure_routing_table
File /usr/local/bundle/gems/neo4j-ruby-driver-4.4.0.beta.1/ruby/neo4j/driver/internal/cluster/routing_table_registry_impl.rb:19 in block in ensure_routing_table

[klobuczek]

Thank you for the report. We know that broken connections are not removed from the pool properly. That's why the following specs are reusing the broken connection. We are working on fixing that. It would be great if you could create a failing spec for the initial error that breaks the connection.

[danielmconrad]

We've pulled in the latest 4.4.0 version and no longer have this issue!