search

antchfx / antch

Antch, a fast, powerful and extensible web crawling & scraping framework for Go
MIT License
260 stars 41 forks source link

Can't set the request header #12

Open liuzeng01 opened 4 years ago

liuzeng01 commented 4 years ago

Sometimes we would build one request that having different request header ,but the antch don't have this setting method .Yeah ,Can we add this function in the antch spider?Just like doing one interface in the crawler settings.

zhengchun commented 4 years ago

you can make a middleware to sets request header, like Compression middleware.

https://github.com/antchfx/antch/blob/master/compression.go