scrapy / protego

A pure-Python robots.txt parser with support for modern conventions.

BSD 3-Clause "New" or "Revised" License

54 stars 28 forks source link

robots.txt can now be read as bytes #18

Closed Eeelco closed 2 years ago

Eeelco commented 2 years ago

This should fix #16

codecov[bot] commented 2 years ago

Codecov Report

Merging #18 (41cf796) into master (11ee491) will not change coverage. The diff coverage is n/a.

:exclamation: Current head 41cf796 differs from pull request most recent head 5eb472f. Consider uploading reports for the commit 5eb472f to get more accurate results

@@     Coverage Diff      @@
##   master   #18   +/-   ##
============================
============================

Gallaecio commented 2 years ago

Could you include a test for the change?

Eeelco commented 2 years ago

Sorry for the large number of commits - it might be too early for me. It seems that there are now str/unicode problems in the Python2.7 build. Is there a preferred way to handle this kind of version problems?

Gallaecio commented 2 years ago

I did not realize we still supported Python 2… And Travis CI…

I think we need to migrate to GitHub Actions and we probably should also remove support for Python 2. I’ll eventually handle that in a separate pull request.

Regarding this pull request, it looks good to me, we just need to delay merging until we handle that other part first.

Gallaecio commented 2 years ago

Closing and reopening to trigger a new run after merging https://github.com/scrapy/protego/pull/25

Gallaecio commented 2 years ago

Oh. I cannot reopen, the source repository has been deleted.

Laerte commented 2 years ago

@Gallaecio Can i open a PR with this change and put @Eeelco as co-author?

Gallaecio commented 2 years ago

That would be great!