The same issue described here: https://github.com/ostinelli/apnotic/issues/68 started biting us in production since this morning. It seems Apple changed something in production since this particular platform has been working flawlessly in production for months.
In our case this had the effect of all our Sidekiq jobs crashing and our queues being filled up.
The error we were getting was:
SocketError: Socket was remotely closed
from /home/deploy/rails_app/shared/bundle/ruby/2.3.0/gems/net-http2-0.15.0/lib/net-http2/client.rb:122:in `callback_or_raise'
from /home/deploy/rails_app/shared/bundle/ruby/2.3.0/gems/net-http2-0.15.0/lib/net-http2/client.rb:107:in `rescue in block (2 levels) in ensure_open'
from /home/deploy/rails_app/shared/bundle/ruby/2.3.0/gems/net-http2-0.15.0/lib/net-http2/client.rb:101:in `block (2 levels) in ensure_open'
As described in https://github.com/ostinelli/apnotic/issues/68, the fix has already been merged. Upgrading to latest version of this gem from master upgraded the gem and its underlying http libraries and fixed the issue immediately:
bundle update --source apnotic
...
Using http-2 0.9.0 (was 0.8.2)
Fetching net-http2 0.18.0 (was 0.15.0)
Installing net-http2 0.18.0 (was 0.15.0)
Using apnotic 1.4.0 (was 1.1.0) from git://github.com/ostinelli/apnotic.git (at master@fc9eb88)
...
Opening this issue so it will be easy to find by anyone who is currently being bitten by this.
The same issue described here: https://github.com/ostinelli/apnotic/issues/68 started biting us in production since this morning. It seems Apple changed something in production since this particular platform has been working flawlessly in production for months.
In our case this had the effect of all our Sidekiq jobs crashing and our queues being filled up.
The error we were getting was:
As described in https://github.com/ostinelli/apnotic/issues/68, the fix has already been merged. Upgrading to latest version of this gem from master upgraded the gem and its underlying http libraries and fixed the issue immediately:
Opening this issue so it will be easy to find by anyone who is currently being bitten by this.