-
why Post method need a map for form data and the key and value not encoding ?
e.g # not replace with %23
-
Hi,
According to [colly documentation](http://go-colly.org/docs/best_practices/distributed/#distributed-scrapers) distributed scrapers
```
the best you can do is wrapping the scraper in a server. …
-
Need to be done:
- https://stackoverflow.com/questions/27231113/999-error-code-on-head-request-to-linkedin
- good alternative https://github.com/raviqqe/muffet
- make it more productive
-
Quick question.
I need to scrape documentation from web sites. After I have grabbed the documentation from their web site I need to run jobs to go back everyday and only get anything new.
Do you…
ghost updated
6 years ago
-
And I'm too lazy to make a pull request for that, could you be so kind to add it in the list?
There is a link: https://github.com/altsab/gowap
-
One of the sites I'm trying this on requires some data to be input and then a page is dynamically loaded. However, the url remains exactly the same always.
For example, one input field is a drop…
-
Hello,
There is a blacklist of domains, but not a blacklist of URL regexp's.
I tried to implement it, I thinks I do it well, but it don't works. D:
This is my code: https://github.com/OSPG/colly/…
-
I am passing information between collectors and setting scraped data using the context. Actually i do something like ctx.Put("collector.From",X) and ctx.Put("object.Name",X). This adds some overheat t…
-
I'm relatively new to golang so while I realize the url parsing is done using golang's core libraries, I have still found an issue that may be valuable to solve in a crawling project.
If you look a…
-
I'm having a slight issue with app engine and wondering whether I've missed some config stage to get colly working correctly. Sorry if this is the wrong place to be posting this.
I created my code …