If you use HttpAuthMiddleware (i.e. the http_user and http_pass spider attributes) for Splash authentication, any non-Splash request will expose your credentials to the request target. This includes robots.txt requests sent by Scrapy when the ROBOTSTXT_OBEY setting is set to True.
Use the new SPLASH_USER and SPLASH_PASS settings instead to set your Splash authentication credentials safely.
Responses now expose the HTTP status code and headers from Splash as response.splash_response_status and response.splash_response_headers (#158)
The meta argument passed to the scrapy_splash.request.SplashRequest constructor is no longer modified (#164)
Website responses with 400 or 498 as HTTP status code are no longer handled as the equivalent Splash responses (#158)
Cookies are no longer sent to Splash itself (#156)
scrapy_splash.utils.dict_hash now also works with obj=None (225793b)
Our test suite now includes integration tests (#156) and tests can be run in parallel (6fb8c41)
There’s a new ‘Getting help’ section in the README.rst file (#161, #162), the documentation about SPLASH_SLOT_POLICY has been improved (#157) and a typo as been fixed (#121)
Made some internal improvements (ee5000d, 25de545, 2aaa79d)
If you use HttpAuthMiddleware_ (i.e. the http_user and http_pass
spider attributes) for Splash authentication, any non-Splash request will
expose your credentials to the request target. This includes robots.txt
requests sent by Scrapy when the ROBOTSTXT_OBEY setting is set to
True.
Use the new SPLASH_USER and SPLASH_PASS settings instead to set
your Splash authentication credentials safely.
Responses now expose the HTTP status code and headers from Splash as
response.splash_response_status and
response.splash_response_headers (#158)
The meta argument passed to the scrapy_splash.request.SplashRequest
constructor is no longer modified (#164)
Website responses with 400 or 498 as HTTP status code are no longer
handled as the equivalent Splash responses (#158)
Cookies are no longer sent to Splash itself (#156)
scrapy_splash.utils.dict_hash now also works with obj=None
(225793b)
Our test suite now includes integration tests (#156) and tests can be run
in parallel (6fb8c41)
There’s a new ‘Getting help’ section in the README.rst file (#161,
#162), the documentation about SPLASH_SLOT_POLICY has been improved
(#157) and a typo as been fixed (#121)
Made some internal improvements (ee5000d, 25de545, 2aaa79d)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language
You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/entrepreneur-interet-general/OpenScraper/network/alerts).
Bumps scrapy-splash from 0.7.2 to 0.8.0.
Release notes
Sourced from scrapy-splash's releases.
Changelog
Sourced from scrapy-splash's changelog.
Commits
b42e191README.rst: remove :ref: usagee942d28CHANGES.rst: update 0.8.0 release date8643db2CHANGES.rst: remove :ref: usage263418aGitHub Actions: use the official codecov action (#293)2b253e5Implement SPLASH_USER and SPLASH_PASS783c58dVersion bump for 0.8.0 (#230)c96e00bTravis CI → GitHub Actions (#292)e40ca4fTST test for GH-164f05c870Merge pull request #164 from whalebot-helmsman/master6658431use a copy instead of changing external dictDependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting
@dependabot rebase.Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/entrepreneur-interet-general/OpenScraper/network/alerts).