temoto / robotstxt

The robots.txt exclusion protocol implementation for Go language
MIT License
269 stars 55 forks source link

Expose rules Api #26

Closed jakesylvestre closed 4 years ago

jakesylvestre commented 4 years ago

This allows for iterating over rules to produce output with parsed data

codecov[bot] commented 4 years ago

Codecov Report

Merging #26 into master will decrease coverage by 0.60%. The diff coverage is 0.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master      #26      +/-   ##
==========================================
- Coverage   92.07%   91.47%   -0.61%     
==========================================
  Files           3        3              
  Lines         303      305       +2     
==========================================
  Hits          279      279              
- Misses         13       15       +2     
  Partials       11       11              
Impacted Files Coverage Δ
robotstxt.go 91.86% <0.00%> (-2.19%) :arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 87d22cd...b9a219c. Read the comment docs.

temoto commented 4 years ago

Do you need help?

jakesylvestre commented 4 years ago

@temoto This was an attempt to expose rules so they were iterable for a quick script I was writing. Decided about halfway through that my use case probably wasn't common enough to justify the expansion in scope. If you want to take a look at what I'm using, there's a diff here

temoto commented 4 years ago

@jakesyl