Return NotImplemented from ErrorDetails.__ne__. #8538
Don't evaluate DateTimeField.default_timezone when a custom timezone is set. #8531
Make relative URLs clickable in Browseable API. #8464
Support ManyRelatedField falling back to the default value when the attribute specified by dot notation doesn't exist. Matches ManyRelatedField.get_attribute to Field.get_attribute. #7574
:cve:CVE-2018-17175: Fix behavior when an empty list is passed as the only argument
(:issue:772). Thanks :user:deckar01 for reporting and thanks
:user:lafrech for the fix.
2.15.0 (2017-12-02)
+++++++++++++++++++
Bug fixes:
Handle UnicodeDecodeError when deserializing bytes with a
String field (:issue:650). Thanks :user:dan-blanchard for the
suggestion and thanks :user:4lissonsilveira for the PR.
2.14.0 (2017-10-23)
+++++++++++++++++++
Features:
Add require_tld parameter to validate.URL (:issue:664).
Thanks :user:sduthil for the suggestion and the PR.
2.13.6 (2017-08-16)
+++++++++++++++++++
Bug fixes:
Fix serialization of types that implement __getitem__
(:issue:669). Thanks :user:MichalKononenko.
2.13.5 (2017-04-12)
+++++++++++++++++++
Bug fixes:
Fix validation of iso8601-formatted dates (:issue:556). Thanks :user:lafrech for reporting.
2.13.4 (2017-03-19)
+++++++++++++++++++
Bug fixes:
Fix symmetry of serialization and deserialization behavior when passing a dot-delimited path to the attribute parameter of fields (:issue:450). Thanks :user:itajaja for reporting.
NumPy 1.22.0 is a big release featuring the work of 153 contributors
spread over 609 pull requests. There have been many improvements,
highlights are:
Annotations of the main namespace are essentially complete. Upstream
is a moving target, so there will likely be further improvements,
but the major work is done. This is probably the most user visible
enhancement in this release.
A preliminary version of the proposed Array-API is provided. This is
a step in creating a standard collection of functions that can be
used across application such as CuPy and JAX.
NumPy now has a DLPack backend. DLPack provides a common interchange
format for array (tensor) data.
New methods for quantile, percentile, and related functions. The
new methods provide a complete set of the methods commonly found in
the literature.
A new configurable allocator for use by downstream projects.
These are in addition to the ongoing work to provide SIMD support for
commonly used functions, improvements to F2PY, and better documentation.
The Python versions supported in this release are 3.8-3.10, Python 3.7
has been dropped. Note that 32 bit wheels are only provided for Python
3.8 and 3.9 on Windows, all other wheels are 64 bits on account of
Ubuntu, Fedora, and other Linux distributions dropping 32 bit support.
All 64 bit wheels are also linked with 64 bit integer OpenBLAS, which should fix
the occasional problems encountered by folks using truly huge arrays.
Expired deprecations
Deprecated numeric style dtype strings have been removed
Using the strings "Bytes0", "Datetime64", "Str0", "Uint32",
and "Uint64" as a dtype will now raise a TypeError.
Expired deprecations for loads, ndfromtxt, and mafromtxt in npyio
numpy.loads was deprecated in v1.15, with the recommendation that
users use pickle.loads instead. ndfromtxt and mafromtxt were both
deprecated in v1.17 - users should use numpy.genfromtxt instead with
the appropriate value for the usemask parameter.
Relaxed the restriction introduced in 2.6.2 so that the Proxy-Authentication header can again be set explicitly in certain cases, restoring compatibility with scrapy-zyte-smartproxy 2.1.0 and older
The Authorization header is now dropped on redirects to a different
scheme (http:// or https://) or port, even if the domain is the
same. Please, see the 4qqq-9vqf-3h3f security advisory_ for more
information.
When using system proxy settings that are different for http:// and https://, redirects to a different URL scheme will now also trigger the
corresponding change in proxy settings for the redirected request. Please,
see the jm3v-qxmh-hxwv security advisory_ for more information.
(:issue:767)
:attr:Spider.allowed_domains <scrapy.Spider.allowed_domains> is now
enforced for all requests, and not only requests from spider callbacks.
(:issue:1042, :issue:2241, :issue:6358)
:func:~scrapy.utils.iterators.xmliter_lxml no longer resolves XML
entities. (:issue:6265)
defusedxml_ is now used to make
:class:scrapy.http.request.rpc.XmlRpcRequest more secure.
(:issue:6250, :issue:6251)
- Restored support for brotlipy_, which had been dropped in Scrapy 2.11.1 in
favor of brotli_. (:issue:`6261`)
.. _brotli: https://github.com/google/brotli
.. note:: brotlipy is deprecated, both in Scrapy and upstream. Use brotli
instead if you can.
</tr></table>
</code></pre>
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/scrapy/scrapy/commit/e8cb5a03b382b98f2c8945355076390f708b918d"><code>e8cb5a0</code></a> Bump version: 2.11.1 → 2.11.2</li>
<li><a href="https://github.com/scrapy/scrapy/commit/2c031f4061ae9bf486cc9e2a699355450638e8c2"><code>2c031f4</code></a> Set the release date of 2.11.2</li>
<li><a href="https://github.com/scrapy/scrapy/commit/3ffa17c0204deb3bdf2c7c60f5a56c9f777698c6"><code>3ffa17c</code></a> Use posargs for pypy3-pinned</li>
<li><a href="https://github.com/scrapy/scrapy/commit/c6a8f0e4d945622a7e71adf635e272b66eddbbd0"><code>c6a8f0e</code></a> Update VERSION references</li>
<li><a href="https://github.com/scrapy/scrapy/commit/60d2577284128cd0cf4af54745730da4a9005177"><code>60d2577</code></a> Merge remote-tracking branch '23j4/2.11.2-release-notes' into 2.11</li>
<li><a href="https://github.com/scrapy/scrapy/commit/36287cb665ab4b0c65fd53181c9a0ef04990ada6"><code>36287cb</code></a> Merge branch 'redirect-protocols' into 2.11</li>
<li><a href="https://github.com/scrapy/scrapy/commit/f138d5d1450ef38ee077c2472c136c70d8d673e8"><code>f138d5d</code></a> Merge branch 'environ-proxy-protocol' into 2.11</li>
<li><a href="https://github.com/scrapy/scrapy/commit/1d0502f25bbe55a22899af915623fda1aaeb9dd8"><code>1d0502f</code></a> Merge branch 'advisory-fix' into 2.11</li>
<li><a href="https://github.com/scrapy/scrapy/commit/bb948af00babe545a7fb52700f4ba1424d206677"><code>bb948af</code></a> Release notes for 2.11.2 (<a href="https://redirect.github.com/scrapy/scrapy/issues/6359">#6359</a>)</li>
<li><a href="https://github.com/scrapy/scrapy/commit/5ad9433dd59cd8436ce33bf2c44796516eef4c3c"><code>5ad9433</code></a> Merge remote-tracking branch 'scrapy/2.11' into 2.11</li>
<li>Additional commits viewable in <a href="https://github.com/scrapy/scrapy/compare/1.6.0...2.11.2">compare view</a></li>
</ul>
</details>
<br />
A crash was resolved when using iterwalk() (or canonicalize())
after parsing certain incorrect input. Note that iterwalk() can crash
on valid input parsed with the same parser after failing to parse the
incorrect input.
4.9.0 (2022-06-01)
Bugs fixed
GH#341: The mixin inheritance order in lxml.html was corrected.
Patch by xmo-odoo.
Other changes
Built with Cython 0.29.30 to adapt to changes in Python 3.11 and 3.12.
Wheels include zlib 1.2.12, libxml2 2.9.14 and libxslt 1.1.35
(libxml2 2.9.12+ and libxslt 1.1.34 on Windows).
GH#343: Windows-AArch64 build support in Visual Studio.
Patch by Steve Dower.
4.8.0 (2022-02-17)
Features added
GH#337: Path-like objects are now supported throughout the API instead of just strings.
Patch by Henning Janssen.
The ElementMaker now supports QName values as tags, which always override
the default namespace of the factory.
Bugs fixed
GH#338: In lxml.objectify, the XSI float annotation "nan" and "inf" were spelled in
lower case, whereas XML Schema datatypes define them as "NaN" and "INF" respectively.
... (truncated)
Commits
d01872c Prevent parse failure in new test from leaking into later test runs.
If you use HttpAuthMiddleware (i.e. the http_user and http_pass spider attributes) for Splash authentication, any non-Splash request will expose your credentials to the request target. This includes robots.txt requests sent by Scrapy when the ROBOTSTXT_OBEY setting is set to True.
Use the new SPLASH_USER and SPLASH_PASS settings instead to set your Splash authentication credentials safely.
Responses now expose the HTTP status code and headers from Splash as response.splash_response_status and response.splash_response_headers (#158)
The meta argument passed to the scrapy_splash.request.SplashRequest constructor is no longer modified (#164)
Website responses with 400 or 498 as HTTP status code are no longer handled as the equivalent Splash responses (#158)
Cookies are no longer sent to Splash itself (#156)
scrapy_splash.utils.dict_hash now also works with obj=None (225793b)
Our test suite now includes integration tests (#156) and tests can be run in parallel (6fb8c41)
There’s a new ‘Getting help’ section in the README.rst file (#161, #162), the documentation about SPLASH_SLOT_POLICY has been improved (#157) and a typo as been fixed (#121)
Made some internal improvements (ee5000d, 25de545, 2aaa79d)
If you use HttpAuthMiddleware_ (i.e. the http_user and http_pass
spider attributes) for Splash authentication, any non-Splash request will
expose your credentials to the request target. This includes robots.txt
requests sent by Scrapy when the ROBOTSTXT_OBEY setting is set to
True.
Use the new SPLASH_USER and SPLASH_PASS settings instead to set
your Splash authentication credentials safely.
Responses now expose the HTTP status code and headers from Splash as
response.splash_response_status and
response.splash_response_headers (#158)
The meta argument passed to the scrapy_splash.request.SplashRequest
constructor is no longer modified (#164)
Website responses with 400 or 498 as HTTP status code are no longer
handled as the equivalent Splash responses (#158)
Cookies are no longer sent to Splash itself (#156)
scrapy_splash.utils.dict_hash now also works with obj=None
(225793b)
Our test suite now includes integration tests (#156) and tests can be run
in parallel (6fb8c41)
There’s a new ‘Getting help’ section in the README.rst file (#161,
#162), the documentation about SPLASH_SLOT_POLICY has been improved
(#157) and a typo as been fixed (#121)
Made some internal improvements (ee5000d, 25de545, 2aaa79d)
twisted.python.filepath.FilePath and related classes (twisted.python.filepath.IFilepath, twisted.python.filepath.AbstractFilePath, twisted.python.zippath.ZipPath, and twisted.python.zippath.ZipArchive) now have type annotations. Additionally, FilePath is now generic, describing its mode, so you can annotate variables as FilePath[str] or FilePath[bytes] depending on the types that you wish to get back from the 'path' attribute and related methods like 'basename'. (#11822)
When using CPython, functions wrapped by twisted.internet.defer.inlineCallbacks can have their arguments and return values freed immediately after completion (due to there no longer being circular references). (#11885)
Bugfixes
Fix TypeError on t.i.cfreactor due to 3.10 type annotation syntax (#11965)
Fix the type annotations of DeferredLock.run, DeferredSemaphore.run, maybeDeferred, ensureDeferred, inlineCallbacks and fromCoroutine that used to return Deferred[Any] to return the result of the passed Coroutine/Coroutine function (#11985)
Fixed significant performance overhead (CPU and bandwidth) when doing small writes to a TLS transport. Specifically, small writes to a TLS transport are now buffered until the next reactor iteration. (#11989)
twisted.python.filepath.FilePath and related classes (twisted.python.filepath.IFilepath, twisted.python.filepath.AbstractFilePath, twisted.python.zippath.ZipPath, and twisted.python.zippath.ZipArchive) now have type annotations. Additionally, FilePath is now generic, describing its mode, so you can annotate variables as FilePath[str] or FilePath[bytes] depending on the types that you wish to get back from the 'path' attribute and related methods like 'basename'. (#11822)
When using CPython, functions wrapped by twisted.internet.defer.inlineCallbacks can have their arguments and return values freed immediately after completion (due to there no longer being circular references). (#11885)
Bugfixes
Fix TypeError on t.i.cfreactor due to 3.10 type annotation syntax (#11965)
Fix the type annotations of DeferredLock.run, DeferredSemaphore.run, maybeDeferred, ensureDeferred, inlineCallbacks and fromCoroutine that used to return Deferred[Any] to return the result of the passed Coroutine/Coroutine function (#11985)
Fixed significant performance overhead (CPU and bandwidth) when doing small writes to a TLS transport. Specifically, small writes to a TLS transport are now buffered until the next reactor iteration. (#11989)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency
- `@dependabot ignore major version` will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself)
- `@dependabot ignore minor version` will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself)
- `@dependabot ignore ` will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore ` will remove all of the ignore conditions of the specified dependency
- `@dependabot unignore ` will remove the ignore condition of the specified dependency and ignore conditions
You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/scrapinghub/portia/network/alerts).
Bumps the pip group with 2 updates in the /portia_server directory: djangorestframework and marshmallow. Bumps the pip group with 4 updates in the /slybot directory: numpy, scrapy, lxml and scrapy-splash. Bumps the pip group with 2 updates in the /slyd directory: twisted and autobahn.
Updates
djangorestframeworkfrom 3.7.7 to 3.15.2Release notes
Sourced from djangorestframework's releases.
... (truncated)
Commits
c7a7eaeVersion 3.15.2 (#9439)3b41f01Fix potential XSS vulnerability in break_long_headers template filter (#9435)fe92f0dAdd__hash__method forpermissions.OperandHolderclass (#9417)fbdab09docs: Correct some evaluation results and a httpie option in Tutorial1 (#9421)36d5c0etests: Check urlpatterns after cleanups (#9400)9d4ed05Don't use Windows line endingsb34bde4Fix typo in setup.cfg settingab681f2Update requirements in docs2237724bump pygments (security hygiene)d58b8daUpdate deprecation hintsUpdates
marshmallowfrom 2.8.0 to 2.15.1Changelog
Sourced from marshmallow's changelog.
... (truncated)
Commits
251bff3Bump version and update changelogd5d9cb2Merge pull request #782 from Nobatek/dev_2x_772_only_emptye849fd8Docs: BaseSchema docstring reword98f2b47Don't serialize any field if only is empty.07cebb9Merge pull request #719 from marshmallow-code/code-of-conduct2b94e65Remove incorrect Nested documentationfa91308Add code of conduct and update contributing docs764f504Remove unnecessary deps8a3b3ceUpdate license year and footer link1579398Add missing@post_load decorator in exampleUpdates
numpyfrom 1.16.4 to 1.22.0Release notes
Sourced from numpy's releases.
... (truncated)
Commits
4adc87dMerge pull request #20685 from charris/prepare-for-1.22.0-releasefd66547REL: Prepare for the NumPy 1.22.0 release.125304bwipc283859Merge pull request #20682 from charris/backport-204165399c03Merge pull request #20681 from charris/backport-20954f9c45f8Merge pull request #20680 from charris/backport-20663794b36fUpdate armccompiler.pyd93b14eUpdate test_public_api.py7662c07Update init.py311ab52Update armccompiler.pyUpdates
scrapyfrom 1.6.0 to 2.11.2Release notes
Sourced from scrapy's releases.
... (truncated)
Changelog
Sourced from scrapy's changelog.