Redirect not working correctly.

[davidlinketech]

First of all: "tls_client.WithNotFollowRedirects()," - somehow has the effect, that it follows redirects.

Furthermore when I'm trying to get "https://httpbin.org/cookies/set/name/value" it's stuck in an infinitive redirect loop. This is because its not getting the new URL from the "Location"-header but instead just the old URL.

[bogdanfinn]

Hey @davidlinketech, thank you for reporting but somehow i'm not able to reproduce your issue.

I created two example functions with your provided URL. See Below

When i execute both, then the result is as expected:

➜ go run with-redirect.go && go run without-redirect.go
2022/11/12 00:21:56 GET https://httpbin.org/cookies/set/name/value (With Follow Redirects): 200
2022/11/12 00:21:57 GET https://httpbin.org/cookies/set/name/value (With Not Follow Redirects) : 302

Here are two functions, one with automatic redirect one without:

without-redirect.go

func main() {
    jar, _ := cookiejar.New(nil)
    options := []tls_client.HttpClientOption{
        tls_client.WithTimeout(60),
        tls_client.WithClientProfile(tls_client.Chrome_107),
        tls_client.WithCookieJar(jar),
        tls_client.WithNotFollowRedirects(),
    }

    client, err := tls_client.NewHttpClient(tls_client.NewNoopLogger(), options...)

    u := "https://httpbin.org/cookies/set/name/value"
    req, err := http.NewRequest(http.MethodGet, u, nil)
    if err != nil {
        log.Println(err)
        return
    }

    req.Header = http.Header{
        "accept":          {"*/*"},
        "content-type":    {"text/html"},
        "accept-language": {"de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7"},
        "user-agent":      {"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36"},
        http.HeaderOrderKey: {
            "accept",
            "content-type",
            "accept-language",
            "user-agent",
        },
    }

    resp, err := client.Do(req)

    if err != nil {
        log.Println(err)
        return
    }

    defer resp.Body.Close()

    log.Println(fmt.Sprintf("GET %s (With Not Follow Redirects) : %d", u, resp.StatusCode))
}

with-redirect.go

func main() {
    jar, _ := cookiejar.New(nil)
    options := []tls_client.HttpClientOption{
        tls_client.WithTimeout(60),
        tls_client.WithClientProfile(tls_client.Chrome_107),
        tls_client.WithCookieJar(jar),
    }

    client, err := tls_client.NewHttpClient(tls_client.NewNoopLogger(), options...)

    u := "https://httpbin.org/cookies/set/name/value"
    req, err := http.NewRequest(http.MethodGet, u, nil)
    if err != nil {
        log.Println(err)
        return
    }

    req.Header = http.Header{
        "accept":          {"*/*"},
        "content-type":    {"text/html"},
        "accept-language": {"de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7"},
        "user-agent":      {"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36"},
        http.HeaderOrderKey: {
            "accept",
            "content-type",
            "accept-language",
            "user-agent",
        },
    }

    resp, err := client.Do(req)

    if err != nil {
        log.Println(err)
        return
    }

    defer resp.Body.Close()

    log.Println(fmt.Sprintf("GET %s (With Follow Redirects): %d", u, resp.StatusCode))
}

[davidlinketech]

Thanks for the quick answer! Its really weird. With Follow Redirects everything works fine, but when I try to stop following redirects, the client does stop following redirects, but the api, in which I'm using the client, doesnt stop and basically tries and tries again till it doesnt get a 302 response (which will never be the case). Seems like its not an issue of the client, but rather something in my tls api. Somewhere in these two functions must be the cause of the problem:

http.HandleFunc("/", handleReq)
err := http.ListenAndServe(":"+string(*port), nil)

[bogdanfinn]

@davidlinketech okay so let me sort that out we are talking about 3 components you are using in here:

1) A HTTP Client you use (probably in a different language) to request your own tls-api 2) your tls-api which useses my tls-client to do further requests (just some kind of request forwarding i assume?) 3) my tls-client

Is that correct?

Interesting for me to see would be the content of handleReq, so the actual implementation to see if you are doing something wrong there. Next to it it would be helpful to see how you instantiate my tls client within your api and how you "stop following redirects" (it sounds like you are switching on the fly?)

Just a hint: I implemented also a api application which does just this forwarding stuff here: https://github.com/bogdanfinn/tls-client-api. I mean you do not have to use it, but for my api the swichting of the redirect behavior does work as expected.

[bogdanfinn]

@davidlinketech And i forgot: If my assumptions are correct, make also sure that your HTTP Client you are using to request your own api is also not following redirects.

[davidlinketech]

@davidlinketech And i forgot: If my assumptions are correct, make also sure that your HTTP Client you are using to request your own api is also not following redirects.

This solved it for me, thanks a lot! Didn't thought about it, but of course I have to set allow redirects to false in my other http client as well. Sorry for the inconveniences.

[bogdanfinn]

@davidlinketech glad that its working now :-)