search

filiph / linkcheck

Fast link checker
https://pub.dartlang.org/packages/linkcheck
MIT License
425 stars 51 forks source link

Proposal to integrate the `robots_txt` package. #113

Open vxern opened 1 year ago

vxern commented 1 year ago

Hi, author of the robots_txt package here. I thought that, since linkcheck currently features its own implementation of a parser for robots.txt, it would be a good idea to instead depend on something more complete, tested and documented.

filiph commented 1 year ago

Thanks for the suggestion, @vxern! I'm afraid I don't have the time to implement this. Are you suggesting you'd implement this as well?

Also, cc @parlough for his opinion.

vxern commented 1 year ago

Hi, @filiph. Sure, I wouldn't mind. Let's wait for parlough's opinion first, though. :)

parlough commented 1 year ago

Sounds good to me, but it's a bit hard to determine for sure without seeing what the end result would look like.

I'd be happy to take a look if you open a PR :)