parnurzeal / gorequest

GoRequest -- Simplified HTTP client ( inspired by nodejs SuperAgent )
http://parnurzeal.github.io/gorequest/
MIT License
3.44k stars 414 forks source link

I guess AddCookie/AddCookies doesnt work? #117

Open MikhailKlemin opened 7 years ago

MikhailKlemin commented 7 years ago

Hi!

I just tried to use it vs sites which use cookies to confirm language/currency, and it's just do not work, I am not sure why... I can only confirm that same sites with standard approach (cookiejar and http.Client(jar: cookieJar)) work just fine.

han2015 commented 6 years ago

Hi @MikhailKlemin I just have the same issue, but it works well if i put the AddCookies back the post/get.

kervinson commented 5 years ago

Hi i just have the same issue, i have try many method, but i have not find the way, so i need you help@han2015, this is mine code , func GetGoRequest() { request := gorequest.New() //loginCookie := GetLoginCookie() // //fmt.Println("go request logincookie",loginCookie) cookie := &http.Cookie{Name: "PHPSESSID", Value: "t2cdimadecpoel6i8rcdomk1m6"} fmt.Println("go request cookie--->", cookie) , , err := request. Get("https://test-xxxxxx.cn/api/index.php?r=detail/brokerage-flows&token=cdkqqf1407307954&recommend_id=39ee6603-e648-49b1-12f5-e803efbf383a&devopenid=88374"). Send("r":"detail/recommend-flows","token":"cdkqqf1407307954","recommend_id":"39ee6603-e648-49b1-12f5-e803efbf383a"). AddCookie(cookie). End() if err != nil{ fmt.Printf("%v \n", err) } fmt.Println(request.Header) // this is always map[] } please, give me same help, thank you

han2015 commented 5 years ago

@kervinson correct your Send sentence refer doc(https://github.com/parnurzeal/gorequest/blame/develop/README.md#L120).

go request cookie---> PHPSESSID=t2cdimadecpoel6i8rcdomk1m6
++++++++++++++1
++++++++++++++TypeJSON
++++++++++++++5
[Get https://test-xxxxxx.cn/api/index.php?devopenid=88374&r=detail%2Fbrokerage-flows&recommend_id=39ee6603-e648-49b1-12f5-e803efbf383a&token=cdkqqf1407307954: dial tcp: lookup test-xxxxxx.cn: no such host]
[PHPSESSID=t2cdimadecpoel6i8rcdomk1m6]
evinstar commented 3 years ago

I met the same problem, my code is bvelow

env : window 10, go 1.15, gorequest 1.6.0

package main

import (
    "github.com/parnurzeal/gorequest"
    "net/http"
    "strings"
    "log"
)

func buildCookie(cookie string) []*http.Cookie {
    cookieSlice := make([]*http.Cookie, 0)
    params := strings.Split(cookie, ";")
    for _, line := range params {
        res := strings.Split(line, "=")
        cookieSlice = append(cookieSlice, &http.Cookie{Name: res[0], Value: res[1]})
    }
    return cookieSlice
}

func main() {
    cookies := []string {"cookie1", "cookie2", "cookie3", "cookie4"..}

    r := gorequest.New()
    r = r.Set("Accept","text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8").
            Set("Accept-Encoding","gzip, deflate, br").
            Set("Accept-Language","zh-CN,zh;q=0.8,zh-TW;q=0.7,zh-HK;q=0.5,en-US;q=0.3,en;q=0.2").
            Set("Host","www.tianyancha.com").
            // Set("Referer","https://www.tianyancha.com/search/ocD?base=taiyuan").
            Set("User-Agent","Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Firefox/68.0")
    index := 0

    req, _ := http.NewRequest("GET", "https://www.tianyancha.com/company/3192219802", nil)
    req.Header.Set("Accept","text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8")
    req.Header.Set("Accept-Encoding","gzip, deflate, br")
    req.Header.Set("Accept-Language","zh-CN,zh;q=0.8,zh-TW;q=0.7,zh-HK;q=0.5,en-US;q=0.3,en;q=0.2")
    req.Header.Set("Host","www.tianyancha.com")
    // Set("Referer","https://www.tianyancha.com/search/ocD?base=taiyuan").
    req.Header.Set("User-Agent","Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Firefox/68.0")
    for i := 0; i<3; i ++ {
        // cookies[index%len(cookies)])
        r.AddCookies(buildCookie(cookies[index%len(cookies)]))
        // log.Println(r.Header)
        // r = r.Set("Cookie", cookies[index%len(cookies)])
        response, _, _ := r.Get("https://www.tianyancha.com/company/3192219802").End()
        log.Println("url, ", response.Request.URL.String())

        req.Header.Set("Cookie", cookies[index%len(cookies)])

        client := &http.Client{}
        resp, _ := client.Do(req)
        log.Println(resp.Request.URL.String())
        index ++
    }
}

output

2020/11/11 16:20:42 url,  https://www.tianyancha.com/login?from=https%3A%2F%2Fwww.tianyancha.com%2Fcompany%2F3192219802
2020/11/11 16:20:43 https://www.tianyancha.com/company/3192219802
2020/11/11 16:20:43 url,  https://www.tianyancha.com/login?from=https%3A%2F%2Fwww.tianyancha.com%2Fcompany%2F3192219802
2020/11/11 16:20:43 https://www.tianyancha.com/company/3192219802
2020/11/11 16:20:44 url,  https://www.tianyancha.com/login?from=https%3A%2F%2Fwww.tianyancha.com%2Fcompany%2F3192219802
2020/11/11 16:20:44 https://www.tianyancha.com/company/3192219802

anyway, i cannot get it through via gorequest~~~

evinstar commented 3 years ago

Now I've solved this problem. Its really a misuse of cookie.

When I read source code of gorequest, I found it would clear the agent stat every time before a new request if DoNotClearSuperAgent is default to false.

So the solution is add this code between you set cookie and start a new request.

agent := gorequest.New()
agent.SetDoNotClearSuperAgent(true)