Closed yonatang closed 10 years ago
Don't know, but if you look at http://www.haaretz.co.il/robots.txt, you'll see a reference to it.
On Fri, Aug 8, 2014 at 9:57 PM, Yuval Adam notifications@github.com wrote:
Thanks for the report! Can you point me to pages where this script is called?
— Reply to this email directly or view it on GitHub https://github.com/yuvadm/free-haaretz/issues/4#issuecomment-51643626.
Nice catch. However this URL returns blank for all User-Agent
s I tested. I prefer not to explicitly block any content unless it is required for the paywall.
Thanks for the report! Can you point me to pages where this script is called?