search

wummel / linkchecker

check links in web documents or full websites
http://wummel.github.io/linkchecker/
GNU General Public License v2.0
1.42k stars 234 forks source link

Option to ignore tag: <meta name="robots" content="nofollow"> #806

Open octavsly opened 3 years ago

octavsly commented 3 years ago

If the HTML contains <meta name="robots" content="nofollow"> results in checking only one file. I would like to keep this in my HTML but still check all the links. --no-robots option is not working for this construct. Another option? or reuse the --no-robots?

matkoniecz commented 3 years ago

note that this project is dead, it was revived at https://github.com/linkchecker/linkchecker which may have fixed this issue already