Open atlet opened 2 years ago
This worked perfectly fine for me with go 1.18 Share some repro code. Like a github repo.
package main
import (
"fmt"
"github.com/gocolly/colly/v2"
"github.com/gocolly/redisstorage"
)
func main() {
c := colly.NewCollector()
storage := &redisstorage.Storage{
Address: "localhost:6379",
DB: 0,
Prefix: "job01",
}
err := c.SetStorage(storage)
if err != nil {
panic(err)
}
// On every a element which has href attribute call callback
c.OnHTML("a[href]", func(e *colly.HTMLElement) {
link := e.Attr("href")
// Print link
fmt.Printf("Link found: %q -> %s\n", e.Text, link)
// Visit link found on page
// Only those links are visited which are in AllowedDomains
c.Visit(e.Request.AbsoluteURL(link))
})
// Before making a request print "Visiting ..."
c.OnRequest(func(r *colly.Request) {
fmt.Println("Visiting", r.URL.String())
})
// Start scraping on https://hackerspaces.org
c.Visit("https://hackerspaces.org/")
}
I tried with simple example using colly V2, but I get the following error:
cannot use storage (variable of type *redisstorage.Storage) as storage.Storage value in argument to c.SetStorage: wrong type for method Cookies (have func(u *net/url.URL) string, want func(u *net/url.URL) string)compiler[InvalidIfaceAssign](https://pkg.go.dev/golang.org/x/tools/internal/typesinternal?utm_source%3Dgopls#InvalidIfaceAssign)
Example
Any suggestions how to fix this issue?