search

elliotgao2 / gain

Web crawling framework based on asyncio.
GNU General Public License v3.0
2.03k stars 207 forks source link

Add proxy support #31

Closed babykick closed 7 years ago

babykick commented 7 years ago

Add a proxy setting in spider class and pass it to "session.get" to support fetching through proxy.

elliotgao2 commented 7 years ago

:white_check_mark: SemVer Sentry Result

This PR will not change anything and can be merged freely without breaking backwards compatibility.

If you decide to create a release after merging this PR, you should know that the changes in this PR warrant at least a PATCH version release. If you decide to wait and create a release with more than this PR then the release should be of the biggest change type of all the changes.

SemVer Sentry is in beta, take it's recommendations with a grain of salt and if in doubt, double check. If you want to get in touch with us, use the contact form on our homepage.

See here for more information on semantic versioning (semver).

elliotgao2 commented 7 years ago

Could you please add testing and docs for proxy.