parse localhost with port

chrisakroyd / robots-txt-parser

A lightweight robots.txt parser for Node.js with support for wildcards, caching and promises.

MIT License

12 stars 8 forks source link

Open chantorak opened 1 year ago

chantorak commented 1 year ago

When testing against locally hosted site on localhost getting errors, this is needed to run tests