search

digininja / CeWL

CeWL is a Custom Word List Generator
1.96k stars 258 forks source link

Allow to pass a pattern the url path must match to be followed #57

Closed sudoaza closed 4 years ago

sudoaza commented 4 years ago

Sometimes excludes is not practical (like when crawling a specific user in a platform, ie. github.com/digininja/ or medium.com/@bobytables) this allows to easily constrain the crawling by the url path pattern.

digininja commented 4 years ago

Thanks for this, added a couple of comments to the commit.

sudoaza commented 4 years ago

Fixed (good catch) and added debug message

digininja commented 4 years ago

All looks good.

Bump up the version number at the top of the script and update the readme and I'll get it merged.

sudoaza commented 4 years ago

Bumped version in the script and updated readme, please check i didnt miss anything

digininja commented 4 years ago

Looks good, sorry for the delay, merging now.