How to ignore expired SSL certificates in Colly?

[killernova]

Since I need to visit an unsafe website over https, the Post / Get method will return an error: x509: certificate has expired or is not yet valid. I know I can use InsecureSkipVerify: true when start a request using net / http package, but what should I do if I want to skip SSL certificate check in Colly?

[pukuldua]

Hi @killernova , how you solve this error: x509: certificate has expired or is not yet valid. problem?

[asciimoo]

@pukuldua you can do it by defining a custom http transport for the collector:

import (
    "net/http"
    "crypto/tls"
    "github.com/gocolly/colly"
)

// ...

c := colly.NewCollector()
c.WithTransport(&http.Transport{
    TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
})