Add some improvements - Githubissues

scrapy / protego

A pure-Python robots.txt parser with support for modern conventions.

BSD 3-Clause "New" or "Revised" License

55 stars 28 forks source link

Closed noviluni closed 4 years ago

noviluni commented 4 years ago

I fixed some small things. If you need me to separate the code in different PRs just let me know :)