lox / httpcache

An RFC7234 compliant golang http.Handler for caching HTTP responses
MIT License
264 stars 28 forks source link

Cached requests return empty response #9

Closed bhilton closed 9 years ago

bhilton commented 9 years ago

Using the cli tool with memory or disk-based cache, any request that has been cached returns an empty reply.

To reproduce:

./cli -listen="0.0.0.0:8080" -dir="" -v export http_proxy="http://127.0.0.1:8080/" curl http://getbootstrap.com/css/ curl http://getbootstrap.com/css/

The second curl request returns no data.

lox commented 9 years ago

I get the following:

go run cli/httpcache.go -listen="0.0.0.0:8080" -dir="" -v
2015/01/02 15:03:55 listening on http://0.0.0.0:8080
2015/01/02 15:03:59 GET http://getbootstrap.com/css/ not in shared cache
2015/01/02 15:03:59 passing request upstream
2015/01/02 15:04:00 upstream responded in 1.259198277s
2015/01/02 15:04:00 127.0.0.1 "GET http://getbootstrap.com/css/ HTTP/1.1" (OK) 293474 MISS 1.259437962s
2015/01/02 15:04:00 stored resources [GET:http://getbootstrap.com/css/] in 1.632602ms
2015/01/02 15:04:08 GET http://getbootstrap.com/css/ found in shared cache
2015/01/02 15:04:08 resource has a freshness of 9m49.889332688s
2015/01/02 15:04:08 serving from cache
2015/01/02 15:04:08 resource is 10.110690184s old, updating age from 2
2015/01/02 15:04:08 127.0.0.1 "GET http://getbootstrap.com/css/ HTTP/1.1" (OK) 293474 HIT 1.541573ms
2015/01/02 15:04:12 GET http://getbootstrap.com/css/ found in shared cache
2015/01/02 15:04:12 resource has a freshness of 9m45.918263311s
2015/01/02 15:04:12 serving from cache
2015/01/02 15:04:12 resource is 14.081762935s old, updating age from 2
2015/01/02 15:04:12 127.0.0.1 "GET http://getbootstrap.com/css/ HTTP/1.1" (OK) 293474 HIT 455.408us

and the results seem to be returned ok in curl. Can you reproduce this with the current HEAD?

bhilton commented 9 years ago

Yes, I can still reproduce this. But it seems to be due to the fact that I'm running the cli proxy behind a firewall that uses yet another proxy. The cli tool seems to return sporadic results when fetching against an upstream proxy of its own. When I fetch content directly from my intranet the tool behaves properly.

lox commented 9 years ago

Interesting. Any chance you could run with -dumphttp and post the output either here or in a gist?

bhilton commented 9 years ago

Sure. Here is the dump when curl returns an empty response.

GET /css/ HTTP/1.1 Host: getbootstrap.com Accept: / Proxy-Connection: Keep-Alive User-Agent: curl/7.37.1

2015/01/05 14:27:50 GET http://getbootstrap.com/css/ not in shared cache 2015/01/05 14:27:50 passing request upstream 2015/01/05 14:27:50 upstream responded in 185.90667ms 2015/01/05 14:27:50 resource is uncacheable

<< HTTP/1.1 302 Found << Cache-Control: no-cache << Date: Mon, 05 Jan 2015 22:27:48 GMT << Location: /css/ << Pragma: no-cache << Via: 1.1 XXXXXXXXXXXXXX.com << X-Cache: SKIP

2015/01/05 14:27:50 127.0.0.1 "GET http://getbootstrap.com/css/ HTTP/1.1" (Found) 0 SKIP 186.587862ms

bhilton commented 9 years ago

It seems to point to one of our internal proxies misbehaving. Hard to detect due to round-robin dns. Apologies for the noise, and thanks for the reply!