search

gospider007 / gospider

🚀Gospider is a powerful Golang web crawler that includes all the necessary libraries for transitioning from Python to Golang. It provides a fast and seamless transition for Python web crawlers to Golang.
GNU Lesser General Public License v3.0
91 stars 22 forks source link

Accept-Encoding的问题 #4

Closed eddylin83 closed 1 year ago

eddylin83 commented 1 year ago

最近get一个网站的时候发现出现403，查了一下，发现resp返回Vary是Accept-encoding。用检测发送头的网站看了一下，好像req里面的Accept-encoding只有gzip。而网站要求的是要带br。看了一下代码，默认的Accept-encoding是"gzip, deflate, br"，不知道为什么在https://www.whatismybrowser.com/detect/what-http-headers-is-my-browser-sending 检测只有gzip。操作系统是win10，是否跟安装的库有关？

eddylin83 commented 1 year ago

嗯，主要是不熟悉golang的配置，把ClientOption的DisCompression设置为true就可以了。

gospider007 commented 1 year ago

gospider 的問題，Accept-encoding 拼写错误

gospider007 commented 1 year ago

已修复