search

antchfx / htmlquery

htmlquery is golang XPath package for HTML query.
https://github.com/antchfx/xpath
MIT License
723 stars 73 forks source link

LoadURL sometimes can not get the right page #33

Closed ToniDuanKroos closed 3 years ago

ToniDuanKroos commented 4 years ago

seggest add the follow code: req, _ := http.NewRequest("GET", url, nil) req.Header.Set("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3776.0 Safari/537.36") client := &http.Client{Timeout: time.Second * 5} resp, err := client.Do(req)