search

infinilabs / crawler

🕷️ An easy-to-use spider written in Golang. (previous named GOPA.)
Other
305 stars 82 forks source link

crawling https #38

Closed tzipora2006 closed 5 years ago

tzipora2006 commented 5 years ago

i've tried gopa over https site and failed. is there a way to work around it or some kind of limitation on https?

medcl commented 5 years ago

what's the error message, there should be no difference for https, or can you give me a test url

medcl commented 5 years ago

@tzipora2006 just found the issue with https, maybe the same to yours, it is due to the relative link was not proper resolved. just pushed the fix to the master.