search

chrisakroyd / robots-txt-parser

A lightweight robots.txt parser for Node.js with support for wildcards, caching and promises.
MIT License
12 stars 8 forks source link

Project maintained? #14

Closed abhisheksurve45 closed 1 year ago

abhisheksurve45 commented 1 year ago

Is this project currently maintained?

If not, what could be the best approach to have a robots txt parser?

Google has released its parser https://github.com/google/robotstxt/tree/master however is in C++ :-(

Looking for something in Go,Java or JS.

chrisakroyd commented 1 year ago

Will work on maintenance either this week or next weekend, been meaning to do the bug fix pass so I'll take your issue as inspiration.

abhisheksurve45 commented 1 year ago

Thanks @chrisakroyd