joschi
commented
9 years ago Thanks for reporting this (and including all relevant information)!
In previous versions of Graylog including up to 1.1.4 there's a hard-coded time limit of 61 seconds (don't ask…) for search requests in the Graylog web interface (or more precisely in the Graylog REST client, see https://github.com/Graylog2/graylog2-server/blob/1.1.4/graylog2-rest-client/src/main/java/org/graylog2/restclient/models/UniversalSearch.java#L113) which cannot be overridden.
Beginning with Graylog 1.2.0, there will be a "proper" configurable Elasticsearch request timeout (Graylog2/graylog2-server#1220).
As a (really ugly) workaround you could use the Graylog REST API directly to run the search queries that time out in the web interface.
If we submit long running searches, it crashes after about 60s with a timeout exception. We want to search over a large relative timerange (14 days) in about 22 indexes with at all 2.2 TB auf data.
Versions:
Graylog 1.1.4 elasticsearch 1.6.0 MongoDB 2.6.7
GUI:
Oh no, something went wrong! (You caused a org.graylog2.restclient.lib.APIException. API call failed GET http://@glserver:9290/search/universal/relative/histogram?filter=_&query=_&range=1209600&interval=day&range_type=relative) Reason: There was a problem with your search. We expected HTTP 200, but got a HTTP -1
Stacktrace show in GUI:
org.graylog2.restclient.lib.ApiClientImpl$ApiRequestBuilder#execute (ApiClientImpl.java:451) org.graylog2.restclient.models.UniversalSearch#dateHistogram (UniversalSearch.java:193) controllers.SearchController#renderSearch (SearchController.java:273) controllers.SearchController#index (SearchController.java:126) Routes$$anonfun$routes$1$$anonfun$applyOrElse$7$$anonfun$apply$459#apply (routes_routing.scala:1659) Routes$$anonfun$routes$1$$anonfun$applyOrElse$7$$anonfun$apply$459#apply (routes_routing.scala:1659) play.core.Router$HandlerInvokerFactory$$anon$4#resultCall (Router.scala:264) play.core.Router$HandlerInvokerFactory$JavaActionInvokerFactory$$anon$15$$anon$1#invocation (Router.scala:255) play.core.j.JavaAction$$anon$1#call (JavaAction.scala:55) play.GlobalSettings$1#call (GlobalSettings.java:67) ...
Graylog-Web-Interface application.log
2015-07-13 16:14:41,856 - [ERROR] - from org.graylog2.restclient.lib.ApiClient in play-akka.actor.default-dispatcher-73 REST call failed java.util.concurrent.TimeoutException: Idle connection timeout to glserver/x.x.x.x:9290 of 60000 ms at com.ning.http.client.providers.netty.timeout.TimeoutTimerTask.expire(TimeoutTimerTask.java:43) ~[com.ning.async-http-client-1.8.14.jar:na] at com.ning.http.client.providers.netty.timeout.IdleConnectionTimeoutTimerTask.run(IdleConnectionTimeoutTimerTask.java:54) ~[com.ning.async-http-client-1.8.14.jar:na] at org.jboss.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:556) ~[io.netty.netty-3.9.8.Final.jar:na] at org.jboss.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:632) ~[io.netty.netty-3.9.8.Final.jar:na] at org.jboss.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:369) ~[io.netty.netty-3.9.8.Final.jar:na] at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) ~[io.netty.netty-3.9.8.Final.jar:na] at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_31]
Can we change the default timeout to avoid such exceptions?