search

chrisakroyd / robots-txt-parser

A lightweight robots.txt parser for Node.js with support for wildcards, caching and promises.
MIT License
12 stars 8 forks source link

Fixes/1.1.0 #3

Closed chrisakroyd closed 3 years ago

chrisakroyd commented 3 years ago