search

nstapelbroek / estate-crawler

Scraping the real estate agencies for up-to-date house listings as soon as they arrive!
BSD 3-Clause "New" or "Revised" License
25 stars 6 forks source link

Bump scrapy from 2.4.1 to 2.5.0 #95

Closed dependabot-preview[bot] closed 3 years ago

dependabot-preview[bot] commented 3 years ago

⚠️ Dependabot Preview has been deactivated ⚠️

This pull request was created by Dependabot Preview, and you've upgraded to Dependabot. This means it won't respond to dependabot commands nor will it be automatically closed if a new version is found.

If you close this pull request, Dependabot will re-create it the next time it checks for updates and everything will work as expected.

Bumps scrapy from 2.4.1 to 2.5.0.

Release notes

Sourced from scrapy's releases.

2.5.0

  • Official Python 3.9 support
  • Experimental HTTP/2 support
  • New get_retry_request() function to retry requests from spider callbacks
  • New headers_received signal that allows stopping downloads early
  • New Response.protocol attribute

See the full changelog

Changelog

Sourced from scrapy's changelog.

Scrapy 2.5.0 (2021-04-06)

Highlights:

  • Official Python 3.9 support
  • Experimental HTTP/2 support <http2>
  • New ~scrapy.downloadermiddlewares.retry.get_retry_request function to retry requests from spider callbacks
  • New ~scrapy.signals.headers_received signal that allows stopping downloads early
  • New Response.protocol <scrapy.http.Response.protocol> attribute

Deprecation removals

  • Removed all code that was deprecated in 1.7.0 <1.7-deprecations> and had not already been removed in 2.4.0 <2.4-deprecation-removals>. (4901)
  • Removed support for the SCRAPY_PICKLED_SETTINGS_TO_OVERRIDE environment variable, deprecated in 1.8.0 <1.8-deprecations>. (4912)

Deprecations

  • The scrapy.utils.py36 module is now deprecated in favor of scrapy.utils.asyncgen. (4900)

New features

  • Experimental HTTP/2 support <http2> through a new download handler that can be assigned to the https protocol in the DOWNLOAD_HANDLERS setting. (1854, 4769, 5058, 5059, 5066)

  • The new scrapy.downloadermiddlewares.retry.get_retry_request function may be used from spider callbacks or middlewares to handle the retrying of a request beyond the scenarios that ~scrapy.downloadermiddlewares.retry.RetryMiddleware supports. (3590, 3685, 4902)

  • The new ~scrapy.signals.headers_received signal gives early access to response headers and allows stopping downloads <topics-stop-response-download>. (1772, 4897)

  • The new Response.protocol <scrapy.http.Response.protocol> attribute gives access to the string that identifies the protocol used to download a response. (4878)

  • Stats <topics-stats> now include the following entries that indicate the number of successes and failures in storing feeds <topics-feed-exports>:

    feedexport/success_count/<storage type>
feedexport/failed_count/<storage type>

    Where <storage type> is the feed storage backend class name, such as ~scrapy.extensions.feedexport.FileFeedStorage or ~scrapy.extensions.feedexport.FTPFeedStorage.

    (3947, 4850)

  • The ~scrapy.spidermiddlewares.urllength.UrlLengthMiddleware spider middleware now logs ignored URLs with INFO logging level <levels> instead of DEBUG, and it now includes the following entry into stats <topics-stats> to keep track of the number of ignored URLs:

    urllength/request_ignored_count

    (5036)

  • The ~scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware downloader middleware now logs the number of decompressed responses and the total count of resulting bytes:

    httpcompression/response_bytes
httpcompression/response_count

... (truncated)

Commits
  • e63188c Bump version: 2.4.1 → 2.5.0
  • a71d6ef 2.5.0 release notes (#5028)
  • 099fb6e Merge pull request #5076 from wRAR/qualname
  • 9e3b868 Use qualname in middleware handling.
  • 8c5a3a5 Merge pull request #5073 from elacuesta/fix-urlencode-doseq-arg
  • 9e7cbc0 Fix type for urlencode's doseq argument
  • 127a858 Merge pull request #5063 from noviluni/update_usageError_message
  • 12f2006 Merge pull request #4799 from GeorgeA92/patch-2
  • f0c8d31 Merge pull request #4694 from Jgaldos/improve-httpstatus-all-meta
  • cc095aa Windows pip installation guide (#4736)
  • Additional commits viewable in compare view


Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

If all status checks pass Dependabot will automatically merge this pull request.

Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language - `@dependabot badge me` will comment on this PR with code to add a "Dependabot enabled" badge to your readme Additionally, you can set the following in your Dependabot [dashboard](https://app.dependabot.com): - Update frequency (including time of day and day of week) - Pull request limits (per update run and/or open at any time) - Automerge options (never/patch/minor, and dev/runtime dependencies) - Out-of-range updates (receive only lockfile updates, if desired) - Security updates (receive only security updates, if desired)
dependabot-preview[bot] commented 3 years ago

One of your CI runs failed on this pull request, so Dependabot won't merge it.

Dependabot will still automatically merge this pull request if you amend it and your tests pass.