search

heroku / s3pository

[DEPRECATED][RETIRED] caching proxy server with s3 backend to speed compiles on heroku
9 stars 0 forks source link

Treat 503 or 504 from a downstream provider gracefully #6

Closed jsimone closed 2 years ago

jsimone commented 12 years ago

Right now it results in a request that hits the 30 second request timeout (on the Heroku platform). Can it be made to fail quicker?

Stack trace:

2012-06-14T12:42:10+00:00 heroku[router]: Error H12 (Request timeout) -> HEAD s3pository.heroku.com/maven-central/org/codehaus/jackson/jackson-core-asl/1.9.2/jackson-core-asl-1.9.2.pom dyno=web.1 queue= wait= service=30000ms status=503 bytes=0
2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository: Request for /maven2/org/codehaus/jackson/jackson-core-asl/1.9.2/jackson-core-asl-1.9.2.pom: Status Code: 504
2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository: java.lang.RuntimeException: Unexpected Final Response Code
2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository:     at com.heroku.s3pository.ProxyService.inspectFinalResponse(ProxyService.scala:326)
2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository:     at com.heroku.s3pository.ProxyService$$anonfun$apply$6.apply(ProxyService.scala:116)
2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository:     at com.heroku.s3pository.ProxyService$$anonfun$apply$6.apply(ProxyService.scala:116)
2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository:     at com.twitter.util.Future$$anonfun$onSuccess$1.apply(Future.scala:436)
2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository:     at com.twitter.util.Future$$anonfun$onSuccess$1.apply(Future.scala:435)
2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository:     at com.twitter.util.Promise$$anonfun$respond$1$$anonfun$apply$1.apply$mcV$sp(Future.scala:759)
2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository:     at com.twitter.util.Monitor$.apply(Monitor.scala:119)
2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository:     at com.twitter.util.Promise$$anonfun$respond$1.apply(Future.scala:759)
2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository:     at com.twitter.util.Promise$$anonfun$respond$1.apply(Future.scala:759)
2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository:     at com.twitter.util.Promise$$anonfun$com$twitter$util$Promise$$respondWithoutChaining$1.apply(Future.scala:770)
2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository:     at com.twitter.util.Promise$$anonfun$com$twitter$util$Promise$$respondWithoutChaining$1.apply(Future.scala:765)
2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository:     at com.twitter.concurrent.IVar$$anonfun$runqs$1.apply$mcV$sp(IVar.scala:155)
2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository:     at com.twitter.concurrent.IVar$Schedule.run(IVar.scala:108)
2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository:     at com.twitter.concurrent.IVar$Schedule.apply(IVar.scala:88)
2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository:     at com.twitter.concurrent.IVar.runqs(IVar.scala:150)
2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository:     at com.twitter.concurrent.IVar.set(IVar.scala:289)
2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository:     at com.twitter.util.Promise.cancel(Future.scala:651)
2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository:     at com.twitter.finagle.channel.ServiceToChannelHandler$$anonfun$com$twitter$finagle$channel$ServiceToChannelHandler$$shutdown$2.apply(ServiceToChannelHandler.scala:83)
2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository:     at com.twitter.finagle.channel.ServiceToChannelHandler$$anonfun$com$twitter$finagle$channel$ServiceToChannelHandler$$shutdown$2.apply(ServiceToChannelHandler.scala:83)
2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository:     at scala.Option.foreach(Option.scala:197)
2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository:     at com.twitter.finagle.channel.ServiceToChannelHandler.com$twitter$finagle$channel$ServiceToChannelHandler$$shutdown(ServiceToChannelHandler.scala:83)
2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository:     at com.twitter.finagle.channel.ServiceToChannelHandler.channelClosed(ServiceToChannelHandler.scala:162)
2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository:     at org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(SimpleChannelHandler.java:113)
2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository:     at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:558)
2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository:     at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:777)
2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository:     at org.jboss.netty.channel.SimpleChannelHandler.channelClosed(SimpleChannelHandler.java:223)
2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository:     at com.twitter.finagle.channel.ChannelSemaphoreHandler.channelClosed(ChannelSemaphoreHandler.scala:125)
2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository:     at org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(SimpleChannelHandler.java:113)
2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository:     at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:558)
2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository:     at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:777)
2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository:     (...more...)
sclasen commented 12 years ago

Yeah I think all the timeout logic might be in the group request path

We can add to single path too

Sent from my iPhone

On Jun 14, 2012, at 5:55 AM, John Simonereply@reply.github.com wrote:

Right now it results in a request that hits the 30 second request timeout (on the Heroku platform). Can it be made to fail quicker?

Stack trace:

2012-06-14T12:42:10+00:00 heroku[router]: Error H12 (Request timeout) -> HEAD s3pository.heroku.com/maven-central/org/codehaus/jackson/jackson-core-asl/1.9.2/jackson-core-asl-1.9.2.pom dyno=web.1 queue= wait= service=30000ms status=503 bytes=0 2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository: Request for /maven2/org/codehaus/jackson/jackson-core-asl/1.9.2/jackson-core-asl-1.9.2.pom: Status Code: 504 2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository: java.lang.RuntimeException: Unexpected Final Response Code 2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository: at com.heroku.s3pository.ProxyService.inspectFinalResponse(ProxyService.scala:326) 2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository: at com.heroku.s3pository.ProxyService$$anonfun$apply$6.apply(ProxyService.scala:116) 2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository: at com.heroku.s3pository.ProxyService$$anonfun$apply$6.apply(ProxyService.scala:116) 2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository: at com.twitter.util.Future$$anonfun$onSuccess$1.apply(Future.scala:436) 2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository: at com.twitter.util.Future$$anonfun$onSuccess$1.apply(Future.scala:435) 2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository: at com.twitter.util.Promise$$anonfun$respond$1$$anonfun$apply$1.apply$mcV$sp(Future.scala:759) 2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository: at com.twitter.util.Monitor$.apply(Monitor.scala:119) 2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository: at com.twitter.util.Promise$$anonfun$respond$1.apply(Future.scala:759) 2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository: at com.twitter.util.Promise$$anonfun$respond$1.apply(Future.scala:759) 2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository: at com.twitter.util.Promise$$anonfun$com$twitter$util$Promise$$respondWithoutChaining$1.apply(Future.scala:770) 2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository: at com.twitter.util.Promise$$anonfun$com$twitter$util$Promise$$respondWithoutChaining$1.apply(Future.scala:765) 2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository: at com.twitter.concurrent.IVar$$anonfun$runqs$1.apply$mcV$sp(IVar.scala:155) 2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository: at com.twitter.concurrent.IVar$Schedule.run(IVar.scala:108) 2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository: at com.twitter.concurrent.IVar$Schedule.apply(IVar.scala:88) 2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository: at com.twitter.concurrent.IVar.runqs(IVar.scala:150) 2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository: at com.twitter.concurrent.IVar.set(IVar.scala:289) 2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository: at com.twitter.util.Promise.cancel(Future.scala:651) 2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository: at com.twitter.finagle.channel.ServiceToChannelHandler$$anonfun$com$twitter$finagle$channel$ServiceToChannelHandler$$shutdown$2.apply(ServiceToChannelHandler.scala:83) 2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository: at com.twitter.finagle.channel.ServiceToChannelHandler$$anonfun$com$twitter$finagle$channel$ServiceToChannelHandler$$shutdown$2.apply(ServiceToChannelHandler.scala:83) 2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository: at scala.Option.foreach(Option.scala:197) 2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository: at com.twitter.finagle.channel.ServiceToChannelHandler.com$twitter$finagle$channel$ServiceToChannelHandler$$shutdown(ServiceToChannelHandler.scala:83) 2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository: at com.twitter.finagle.channel.ServiceToChannelHandler.channelClosed(ServiceToChannelHandler.scala:162) 2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository: at org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(SimpleChannelHandler.java:113) 2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository: at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:558) 2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository: at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:777) 2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository: at org.jboss.netty.channel.SimpleChannelHandler.channelClosed(SimpleChannelHandler.java:223) 2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository: at com.twitter.finagle.channel.ChannelSemaphoreHandler.channelClosed(ChannelSemaphoreHandler.scala:125) 2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository: at org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(SimpleChannelHandler.java:113) 2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository: at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:558) 2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository: at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:777) 2012-06-14T12:42:10+00:00 app[web.1]: WAR [20120614-12:42:10.042] s3pository: (...more...)

Reply to this email directly or view it on GitHub: https://github.com/heroku/s3pository/issues/6

joshwlewis commented 2 years ago

Closing as this service is being retired.