search

sjdirect / abot

Cross Platform C# web crawler framework built for speed and flexibility. Please star this project! +1.
Apache License 2.0
2.25k stars 560 forks source link

Fails to load some websites #201

Closed mohodhaka closed 5 years ago

mohodhaka commented 5 years ago

Hi Steven,

This is a great website crawler.

Recently I come across this website which ABot fails to load.

https://shop.coles.com.au/a/a-national/everything/browse

Do you have any idea how I can scrape this?

Thanks Mohsin

sjdirect commented 5 years ago

Hi Mohsin,

Not nearly enough information for anyone to help you with here. You would need to dig in a big and give some examples of what you see vs what you expect. Closing this ticket but free to reopen or create another one if you have more details..