[Feature Request]: use user-agent according to the selected browser profile

[TheFiordi]

Describe the feature / enhancement and how it would improve things

In order to not use a browser profile and then put in the headers a user-agent different from the profile used, I propose creating a new function tls_client.WithUserAgent(browserProfile) which returns a string that contains the corresponding user-agent.

Even in the examples ( requestToppsAsChrome107Client() ) the user agent used in the headers is not set according to the profile used, so I think this function will help avoiding this error.

Describe how your proposal will work, with code and/or pseudo-code

Using the example in ./examples/main.go

options := []tls_client.HttpClientOption{
        tls_client.WithTimeoutSeconds(30),
        tls_client.WithClientProfile(profiles.Chrome_107), //here chrome 107 is used
        tls_client.WithDebug(),
    }

    client, err := tls_client.NewHttpClient(tls_client.NewNoopLogger(), options...)
    if err != nil {
        log.Println(err)
        return
    }

    req, err := http.NewRequest(http.MethodGet, "https://www.topps.com/", nil)
    if err != nil {
        log.Println(err)
        return
    }

    req.Header = http.Header{
        "accept":                    {"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"},
        "accept-encoding":           {"gzip"},
        "accept-language":           {"de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7"},
        "cache-control":             {"max-age=0"},
        "if-none-match":             {`W/"4d0b1-K9LHIpKrZsvKsqNBKd13iwXkWxQ"`},
        "sec-ch-ua":                 {`"Google Chrome";v="105", "Not)A;Brand";v="8", "Chromium";v="105"`}, <--- and also here
        "sec-ch-ua-mobile":          {"?0"},
        "sec-ch-ua-platform":        {`"macOS"`},
        "sec-fetch-dest":            {"document"},
        "sec-fetch-mode":            {"navigate"},
        "sec-fetch-site":            {"none"},
        "sec-fetch-user":            {"?1"},
        "upgrade-insecure-requests": {"1"},
        "user-agent":                {tls_client.WithUserAgent(profiles.Chrome_107)},  <------ here we set the corresponding user-agent
//Mozilla/5.0 (Macintosh; Intel Mac OS X 13_0_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36
        http.HeaderOrderKey: {
            "accept",
            "accept-encoding",
            "accept-language",
            "cache-control",
            "if-none-match",
            "sec-ch-ua",
            "sec-ch-ua-mobile",
            "sec-ch-ua-platform",
            "sec-fetch-dest",
            "sec-fetch-mode",
            "sec-fetch-site",
            "sec-fetch-user",
            "upgrade-insecure-requests",
            "user-agent",
        },
    }

    resp, err := client.Do(req)
    if err != nil {
        log.Println(err)
        return
    }

// rest of code

[combo23]

This doesn't make much sense as for example user agent for the same chrome version will not be the same on different platforms (windows, linux, mac). So in the end there are many user-agents for the same chrome version

[TheFiordi]

you are right but it doesn't make much sense to use the profile of chrome 107 and the user agent of chrome 105, or am I wrong?

[combo23]

why would you use chrome 105

[bogdanfinn]

you are right but it doesn't make much sense to use the profile of chrome 107 and the user agent of chrome 105, or am I wrong?

The issue here is that i just made a stupid copy & paste issue while creating examples :-D

I share the same opinion like combo. this kind of feature makes no sense as you need to pin down to one single user agent per profile. Next to it we would mix more or less two "domains". First the TLS Fingerprint and Second the Request Headers which should be independent in my opinion in order to give more flexibility.

[TheFiordi]

ook I understand, my suggestion sparked from the examples where the user agent wasn't correct so I thought that it'd be useful to have that function in order to avoid the "copy & paste error". Should I open a PR to correct the examples? :) Anyway this library looks dope, keep up the good work!!

[bogdanfinn]

@TheFiordi feel free to open a PR and fix examples or provide new ones. I'm always grateful when somebody contributes :)