Refactor to use built-in redirection middleware

Gonzih / crabler

Web Crawler for Crabs

https://docs.rs/crabler/

MIT License

92 stars 4 forks source link

Refactor to use built-in redirection middleware #14

Closed Susurrus closed 2 years ago

Susurrus commented 2 years ago

There was a bug in my previous redirection code, and switching to the built-in redirection handler fixes it. I think this makes more sense though it adds an extra HTTP request per fetch (to preserve the Body of the request).