While generating tiles with --fetch-wikidata following was observed:
...
Caused by: java.io.IOException: closed
at java.net.http/jdk.internal.net.http.ResponseSubscribers$HttpResponseInputStream.current(ResponseSubscribers.java:464)
at java.net.http/jdk.internal.net.http.ResponseSubscribers$HttpResponseInputStream.read(ResponseSubscribers.java:508)
at java.base/java.io.BufferedInputStream.fill(BufferedInputStream.java:291)
at java.base/java.io.BufferedInputStream.read1(BufferedInputStream.java:347)
at java.base/java.io.BufferedInputStream.implRead(BufferedInputStream.java:420)
at java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:399)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._loadMore(UTF8StreamJsonParser.java:258)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.parseEscapedName(UTF8StreamJsonParser.java:2102)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.slowParseName(UTF8StreamJsonParser.java:2010)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._parseName(UTF8StreamJsonParser.java:1798)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.nextFieldName(UTF8StreamJsonParser.java:1081)
at com.fasterxml.jackson.databind.deser.std.BaseNodeDeserializer._deserializeContainerNoRecursion(JsonNodeDeserializer.java:536)
at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:100)
at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:25)
at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:323)
at com.fasterxml.jackson.databind.ObjectMapper._readTreeAndClose(ObjectMapper.java:4867)
at com.fasterxml.jackson.databind.ObjectMapper.readTree(ObjectMapper.java:3199)
at com.onthegomap.planetiler.util.Wikidata.parseResults(Wikidata.java:92)
at com.onthegomap.planetiler.util.Wikidata.queryWikidata(Wikidata.java:379)
at com.onthegomap.planetiler.util.Wikidata.flush(Wikidata.java:314)
... 9 more
Caused by: java.io.IOException: /<IP of our server>:33120: GOAWAY received
at java.net.http/jdk.internal.net.http.Http2Connection.handleGoAway(Http2Connection.java:1166)
...
Error was observed several times roughly a week ago, but then never again so far.
To Reproduce
While not able to reproduce the issue with WikiData server(s) and not willing to pound on them too much I've tried a mock:
Setup a dummy nginx powered WikiData API mock with dummy JSON response implemented (as static file but allowing POST), HTTP/2 enabled and http2_max_requests 25; setting
Adjust Planetiler to use a dummy nginx mock server
Run Planetiler with --fetch-wikidata
See the error
Main problem is, that I failed to trigger the error within parseResults() since I'm not sure about what were the conditions on servers:
server running Planetiler, e.g. mainly CPU and network load
WikiData server, e.g. again mainly CPU and network load, possibly scheduled or unscheduled reboot, etc.
Expected behavior
Planetiler is handling all IOException by retrying the requests, that is expected behavior.
Screenshots
N/A
Environment (please complete the following information):
Hardware: some server
OS: Linux/Ubuntu
Java version and distribution: 21 shipped by Ubuntu
Describe the bug
While generating tiles with
--fetch-wikidata
following was observed:Error was observed several times roughly a week ago, but then never again so far.
To Reproduce
While not able to reproduce the issue with WikiData server(s) and not willing to pound on them too much I've tried a mock:
http2_max_requests 25;
setting--fetch-wikidata
Main problem is, that I failed to trigger the error within
parseResults()
since I'm not sure about what were the conditions on servers:Expected behavior
Planetiler is handling all
IOException
by retrying the requests, that is expected behavior.Screenshots
N/A
Environment (please complete the following information):
Additional context
Searching around I found this: https://nuvalence.io/insights/lessons-learned-investigating-goaways-in-grpc-apis/ which may implicate
http2_max_requests
setting as a factor in this. Based on that I've tried to trigger GOAWAY with nginx. Alsoquery.wikidata.org
seems to be running on or behind nginx.