Feature Request: option to silence denied by robots output

filiph / linkcheck

Fast link checker

https://pub.dartlang.org/packages/linkcheck

MIT License

397 stars 51 forks source link

Open mmattel opened 5 years ago

mmattel commented 5 years ago

Standard: Access to these URLs denied by robots.txt, so we couldn't check them.

Feature: --silence-robots creates no output

SuperSandro2000 commented 4 years ago

I would really appreciate this feature as I have quite a lot URLs that are ignored by a robots.txt.