-
**the example:**
```
func main() {
url := "https://httpbin.org/delay/1"
// Instantiate default collector
c := colly.NewCollector(colly.AllowURLRevisit())
// create a request queue wi…
-
I was wondering how to use the proxy functionality with Basic Auth? I've modified the callback function in insert the base64 encoded username and password with the `Proxy-Authorization` header. I …
-
Hi there,
thank you for your amazing job, it's really a great framework!
I'm scratching my head several days but can't understand what's wrong ...
I'm using HTTP Forward Proxy (Squid) and User-Age…
-
```go
package main
import (
"log"
"net/http"
"github.com/gocolly/colly/v2"
)
func main() {
// we have to disable Accept-Encoding: gzip,
// because the remote server might send comp…
-
If a server is broken and returns an invalid or missing content-length, colly reports "unexpected EOF", despite having received data.
Repro:
1. use colly to visit https://regexr.com/39p0t
2. r…
-
I did make Scraping for Amazon Product Titles but Amazon captcha catches my scraper. I tried 10 times with go run main.go(8 times catches me - 2 times I scrapped the product title)
I researched thi…
-
When a server requests an upgrade to a websocket, colly will hang indefinitely regardless of timeout settings.
Here is an example program and domain that demonstrates this problem:
```go
packag…
-
(for a future release) using something like http://go-colly.org/ to be able to input a URL and have it scrape the images of certain dimensions into a web gallery? thinking about if a studio releases a…
-
-
- Just like pyspider, how about add a web UI for Colly which can help user to control their spider.