search

census-instrumentation / opencensus-go

A stats collection and distributed tracing framework
http://opencensus.io
Apache License 2.0
2.06k stars 327 forks source link

Connection issues with vanity import URLs #777

Closed gottwald closed 6 years ago

gottwald commented 6 years ago

I hope this is the right place to report: I repeatedly keep having trouble with the opencensus vanity import URLs. I don't have that problem with other Google vanity import URLs like the ones from GCP (cloud.google.com/go)

Often I get this kind of error messages with random opencensus repos when running go dep:

$ dep ensure
  * "contrib.go.opencensus.io/exporter/stackdriver": unable to deduce repository and source type for "contrib.go.opencensus.io/exporter/stackdriver": unable to read metadata: unable to fetch raw metadata: failed HTTP request to URL "http://contrib.go.opencensus.io/exporter/stackdriver?go-get=1": Get http://contrib.go.opencensus.io/exporter/stackdriver?go-get=1: read tcp 192.168.178.51:40300->216.239.32.21:80: read: connection reset by peer

When I repeat the command a couple times it finally succeeds.

This can be reproduced with curl also:

$ curl http://go.opencensus.io/plugin/ochttp?go-get=1 
curl: (56) Recv failure: Connection reset by peer

$ curl http://go.opencensus.io/plugin/ochttp?go-get=1
curl: (56) Recv failure: Connection reset by peer

$ curl http://go.opencensus.io/plugin/ochttp?go-get=1
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
<meta name="go-import" content="go.opencensus.io git https://github.com/census-instrumentation/opencensus-go">
<meta name="go-source" content="go.opencensus.io https://github.com/census-instrumentation/opencensus-go https://github.com/census-instrumentation/opencensus-go/tree/master{/dir} https://github.com/census-instrumentation/opencensus-go/blob/master{/dir}/{file}#L{line}">
<meta http-equiv="refresh" content="0; url=https://godoc.org/go.opencensus.io/plugin/ochttp">
</head>
<body>
Nothing to see here; <a href="https://godoc.org/go.opencensus.io/plugin/ochttp">see the package on godoc</a>.
</body>
</html>

I also tested all 3 advertised IPs separately which seem to have the same behavior:

$ dig +short go.opencensus.io
216.239.34.21
216.239.36.21
216.239.32.21

Here's a wireshark dump of a failing curl:

No.     Time           Source                Destination           Protocol Length Info
    118 4.647644404    192.168.178.51        216.239.36.21         TCP      74     48134 → 80 [SYN] Seq=0 Win=29200 Len=0 MSS=1460 SACK_PERM=1 TSval=3116519058 TSecr=0 WS=128
    120 4.685761383    216.239.36.21         192.168.178.51        TCP      76     80 → 48134 [SYN, ACK] Seq=0 Ack=1 Win=42408 Len=0 MSS=1380 SACK_PERM=1 TSval=3859838150 TSecr=3116519058 WS=256
    121 4.685807743    192.168.178.51        216.239.36.21         TCP      66     48134 → 80 [ACK] Seq=1 Ack=1 Win=29312 Len=0 TSval=3116519096 TSecr=3859838150
    122 4.685892134    192.168.178.51        216.239.36.21         HTTP     168    GET /plugin/ochttp?go-get=1 HTTP/1.1 
    123 4.723859347    216.239.36.21         192.168.178.51        TCP      62     80 → 48134 [RST] Seq=1 Win=0 Len=0
    124 4.723859887    216.239.36.21         192.168.178.51        TCP      62     80 → 48134 [RST] Seq=1 Win=0 Len=0
    158 4.985749645    216.239.36.21         192.168.178.51        TCP      76     [TCP Retransmission] 80 → 48134 [SYN, ACK] Seq=0 Ack=1 Win=42408 Len=0 MSS=1380 SACK_PERM=1 TSval=3859838449 TSecr=3116519058 WS=256
    159 4.985811965    192.168.178.51        216.239.36.21         TCP      54     48134 → 80 [RST] Seq=1 Win=0 Len=0

All my connections come from Germany just in case this is important. Traceroutes are always successful but I could not reproduce this from a scaleway server connecting from Paris. This might as well be a problem with my provider/network or anything but it's weird that even other google vanity urls work.

rakyll commented 6 years ago

OpenCensus is not using the same server cloud.google.com uses for vanity import path support.

We are hosting our server as an AppEngine app, do you have any consistent issues with AppEngine IP ranges?

We should probably switch to manual scaling if we are not already doing so. Automatic scaling drops the connections if the number of pending requests are above a certain limit.

gottwald commented 6 years ago

Not that I know of. Tested all our own app engine apps successfully but they're all hosted in the EU. Googled for further *.appspot.com domains and tested everything I could find on the first 10 Google search pages. All work on the first try, no connection resets.

Did you already change the scaling? Because currently I cannot reproduce the issue with go.opencensus.io, which was possible for at least the last two weeks or so.

semistrict commented 6 years ago

Sounds like this was resolved by changing the scaling on the AppEngine app.

nhooyr commented 5 years ago

Seeing this issue.

go: finding go.opencensus.io/trace latest
go: finding go.opencensus.io/plugin/ochttp latest
go: finding go.opencensus.io/trace/propagation latest
go: finding go.opencensus.io/plugin latest
../../../pkg/mod/google.golang.org/api@v0.0.0-20181217000635-41dc4b66e69d/transport/http/dial.go:25:2: unknown import path "go.opencensus.io/plugin/ochttp": cannot find module providing package go.opencensus.io/plugin/ochttp
../../../pkg/mod/google.golang.org/api@v0.0.0-20181217000635-41dc4b66e69d/transport/http/internal/propagation/http.go:29:2: unknown import path "go.opencensus.io/trace": cannot find module providing package go.opencensus.io/trace
../../../pkg/mod/google.golang.org/api@v0.0.0-20181217000635-41dc4b66e69d/transport/http/internal/propagation/http.go:30:2: unknown import path "go.opencensus.io/trace/propagation": cannot find module providing package go.opencensus.io/trace/propagation
nhooyr commented 5 years ago

rm -rf $GOPATH/pkg/mod fixed the issue.

christian-kreuzberger-dtx commented 4 years ago

We are having the same issue for a couple of days now using go modules and travis-ci:

go: contrib.go.opencensus.io/exporter/ocagent@v0.4.12: unrecognized import path "contrib.go.opencensus.io/exporter/ocagent" (https fetch: Get https://contrib.go.opencensus.io/exporter/ocagent?go-get=1: read tcp 172.17.0.2:56480->216.239.32.21:443: read: connection reset by peer)

any idea how we can fix this please?