search

mtlevolio / pylinkchecker

standalone and pure python link checker and crawler that traverses a web site and reports errors
Other
34 stars 8 forks source link

Add a strict mode #6

Closed bartdag closed 10 years ago

bartdag commented 11 years ago

If in strict mode, take the url as is. Otherwise, strip it of leading and trailing whitespaces.

In the future, we may do more url transformations if not in strict mode to ensure that we treat the page like a browser.