Closed syllith closed 1 year ago
I figured it out. Turns out it was GZIP encoded. I guess I was write about the encoding being wrong. This is what I did for any future visitors. Great package by the way, fixed my crashing when using the stock http package.
req := fasthttp.AcquireRequest()
resp := fasthttp.AcquireResponse()
defer fasthttp.ReleaseRequest(req)
defer fasthttp.ReleaseResponse(resp)
req.SetRequestURI(uri)
err := fasthttp.Do(req, resp)
if err != nil {
time.Sleep(500 * time.Millisecond)
continue
}
if resp.StatusCode() == fasthttp.StatusOK {
//* Convert from GZIP
var reader io.Reader = bytes.NewReader(resp.Body())
switch string(resp.Header.Peek("Content-Encoding")) {
case "gzip":
gzipReader, err := gzip.NewReader(reader)
defer gzipReader.Close()
if err != nil {
time.Sleep(500 * time.Millisecond)
continue
}
reader = gzipReader
}
Hey there, I am executing a ton of network requests to a website using a GET request. For the most part, everything seems to work fine, however, occasionally I'll get a request that looks like this: The response code is 200 and sometimes it happens a lot back to back, but will occasionally fix itself. Other times it's just a one off and doesn't happen again for a while. It almost looks like incorrect encoding or something? The reason I'm asking this question here is because I'm switching from the standard Go HTTP client and I've never seen this behavior with it, only with fasthttp. Is there perhaps something silly I'm doing that would be causing this? Here's the code I'm using:
Thank you for any assistance.