search

mvukov / rules_ros2

Build ROS 2 with Bazel
Apache License 2.0
85 stars 47 forks source link

Update dependency zstd to v1.5.5 #136

Closed renovate[bot] closed 1 year ago

renovate[bot] commented 1 year ago

Mend Renovate

This PR contains the following updates:

Package Type Update Change
zstd http_archive patch v1.5.2 -> v1.5.5

Release Notes

facebook/zstd (zstd) ### [`v1.5.5`](https://togithub.com/facebook/zstd/releases/tag/v1.5.5): Zstandard v1.5.5 [Compare Source](https://togithub.com/facebook/zstd/compare/v1.5.4...v1.5.5) ### Zstandard v1.5.5 Release Note This is a quick fix release. The primary focus is to correct a rare corruption bug in high compression mode, detected by [@​danlark1](https://togithub.com/danlark1) . The probability to generate such a scenario by random chance is extremely low. It evaded months of continuous fuzzer tests, due to the number and complexity of simultaneous conditions required to trigger it. Nevertheless, [@​danlark1](https://togithub.com/danlark1) from Google shepherds such a humongous amount of data that he managed to detect a reproduction case (corruptions are detected thanks to the checksum), making it possible for [@​terrelln](https://togithub.com/terrelln) to investigate and fix the bug. Thanks !\ While the probability might be very small, corruption issues are nonetheless very serious, so an update to this version is highly recommended, especially if you employ high compression modes (levels 16+). When the issue was detected, there were a number of other improvements and minor fixes already in the making, hence they are also present in this release. Let’s detail the main ones. ##### Improved memory usage and speed for the `--patch-from` mode `V1.5.5` introduces memory-mapped dictionaries, by [@​daniellerozenblit](https://togithub.com/daniellerozenblit), for both posix [#​3486](https://togithub.com/facebook/zstd/pull/3486) and windows [#​3557](https://togithub.com/facebook/zstd/pull/3557). This feature allows `zstd` to memory-map large dictionaries, rather than requiring to load them into memory. This can make a pretty big difference for memory-constrained environments operating patches for large data sets. It's mostly visible under memory pressure, since `mmap` will be able to release less-used memory and continue working. But even when memory is plentiful, there are still measurable memory benefits, as shown in the graph below, especially when the reference turns out to be not completely relevant for the patch. ![mmap_memory_usage](https://user-images.githubusercontent.com/48103643/223875190-1c83877c-d9dc-4b1d-99f3-e441d908b15d.png) This feature is automatically enabled for `--patch-from` compression/decompression when the dictionary is larger than the user-set memory limit. It can also be manually enabled/disabled using `--mmap-dict` or `--no-mmap-dict` respectively. Additionally, [@​daniellerozenblit](https://togithub.com/daniellerozenblit) introduces significant speed improvements for `--patch-from`. An `I/O` optimization in [#​3486](https://togithub.com/facebook/zstd/pull/3486) greatly improves `--patch-from` decompression speed on Linux, typically by `+50%` on large files (~1GB). ![patch-from_IO_optimization](https://user-images.githubusercontent.com/48103643/224113381-f852579f-b403-4038-961d-22dbbc12dd7f.png) Compression speed is also taken care of, with a dictionary-indexing speed optimization introduced in [#​3545](https://togithub.com/facebook/zstd/pull/3545). It wildly accelerates `--patch-from` compression, typically doubling speed on large files (~1GB), sometimes even more depending on exact scenario. ![patch_from_compression_speed_optimization](https://user-images.githubusercontent.com/48103643/224391113-05de7ed3-5c62-4cb4-a4a4-eaa7576aa5b6.png) This speed improvement comes at a slight regression in compression ratio, and is therefore enabled only on non-ultra compression strategies. ##### Speed improvements of middle-level compression for specific scenarios The row-hash match finder introduced in version 1.5.0 for levels 5-12 has been improved in version 1.5.5, enhancing its speed in specific corner-case scenarios. The first optimization ([#​3426](https://togithub.com/facebook/zstd/pull/3426)) accelerates streaming compression using `ZSTD_compressStream` on small inputs by removing an expensive table initialization step. This results in remarkable speed increases for very small inputs. The following scenario measures compression speed of `ZSTD_compressStream` at level 9 for different sample sizes on a linux platform running an i7-9700k cpu. | sample size | `v1.5.4` (MB/s) | `v1.5.5` (MB/s) | improvement | | --- | ----:| ---:| --- | | 100 | 1.4 | 44.8 | x32 | 200 | 2.8 | 44.9 | x16 | 500 | 6.5 | 60.0 | x9.2 | 1K | 12.4 | 70.0 | x5.6 | 2K | 25.0 | 111.3 | x4.4 | 4K | 44.4 | 139.4 | x3.2 | ... | ... | ... | | 1M | 97.5 | 99.4 | +2% The second optimization ([#​3552](https://togithub.com/facebook/zstd/issues/3552)) speeds up compression of incompressible data by a large multiplier. This is achieved by increasing the step size and reducing the frequency of matching when no matches are found, with negligible impact on the compression ratio. It makes mid-level compression essentially inexpensive when processing incompressible data, typically, already compressed data (note: this was already the case for fast compression levels). The following scenario measures compression speed of `ZSTD_compress` compiled with `gcc-9` for a ~10MB incompressible sample on a linux platform running an i7-9700k cpu. | level | `v1.5.4` (MB/s) | `v1.5.5` (MB/s) | improvement | | --- | ----:| ---:| --- | | 3 | 3500 | 3500 | not a row-hash level (control) | 5 | 400 | 2500 | x6.2 | 7 | 380 | 2200 | x5.8 | 9 | 176 | 1880 | x10 | 11 | 67 | 1130 | x16 | 13 | 89 | 89 | not a row-hash level (control) ##### Miscellaneous There are other welcome speed improvements in this package. For example, [@​felixhandte](https://togithub.com/felixhandte) managed to increase processing speed of small files by carefully reducing the nb of system calls ([#​3479](https://togithub.com/facebook/zstd/issues/3479)). This can easily translate into +10% speed when processing a lot of small files in batch. The Seekable format received a bit of care. It's now much faster when splitting data into very small blocks ([#​3544](https://togithub.com/facebook/zstd/issues/3544)). In an extreme scenario reported by [@​P-E-Meunier](https://togithub.com/P-E-Meunier), it improves processing speed by x90. Even for more "common" settings, such as using 4KB blocks on some "normally" compressible data like `enwik`, it still provides a healthy x2 processing speed benefit. Moreover, [@​dloidolt](https://togithub.com/dloidolt) merged an optimization that reduces the nb of `I/O` `seek()` events during reads (decompression), which is also beneficial for speed. The release is not limited to speed improvements, several loose ends and corner cases were also fixed in this release. For a more detailed list of changes, please take a look at the changelog. #### Change Log - fix: fix rare corruption bug affecting the high compression mode, reported by [@​danlark1](https://togithub.com/danlark1) ([#​3517](https://togithub.com/facebook/zstd/issues/3517), [@​terrelln](https://togithub.com/terrelln)) - perf: improve mid-level compression speed ([#​3529](https://togithub.com/facebook/zstd/issues/3529), [#​3533](https://togithub.com/facebook/zstd/issues/3533), [#​3543](https://togithub.com/facebook/zstd/issues/3543), [@​yoniko](https://togithub.com/yoniko) and [#​3552](https://togithub.com/facebook/zstd/issues/3552), [@​terrelln](https://togithub.com/terrelln)) - lib: deprecated bufferless block-level API ([#​3534](https://togithub.com/facebook/zstd/issues/3534)) by [@​terrelln](https://togithub.com/terrelln) - cli: `mmap` large dictionaries to save memory, by [@​daniellerozenblit](https://togithub.com/daniellerozenblit) - cli: improve speed of `--patch-from` mode (~+50%) ([#​3545](https://togithub.com/facebook/zstd/issues/3545)) by [@​daniellerozenblit](https://togithub.com/daniellerozenblit) - cli: improve i/o speed (~+10%) when processing lots of small files ([#​3479](https://togithub.com/facebook/zstd/issues/3479)) by [@​felixhandte](https://togithub.com/felixhandte) - cli: `zstd` no longer crashes when requested to write into write-protected directory ([#​3541](https://togithub.com/facebook/zstd/issues/3541)) by [@​felixhandte](https://togithub.com/felixhandte) - cli: fix decompression into block device using `-o` ([#​3584](https://togithub.com/facebook/zstd/issues/3584), [@​Cyan4973](https://togithub.com/Cyan4973)) reported by [@​georgmu](https://togithub.com/georgmu) - build: fix zstd CLI compiled with lzma support but not zlib support ([#​3494](https://togithub.com/facebook/zstd/issues/3494)) by [@​Hello71](https://togithub.com/Hello71) - build: fix `cmake` does no longer require 3.18 as minimum version ([#​3510](https://togithub.com/facebook/zstd/issues/3510)) by [@​kou](https://togithub.com/kou) - build: fix MSVC+ClangCL linking issue ([#​3569](https://togithub.com/facebook/zstd/issues/3569)) by [@​tru](https://togithub.com/tru) - build: fix zstd-dll, version of zstd CLI that links to the dynamic library ([#​3496](https://togithub.com/facebook/zstd/issues/3496)) by [@​yoniko](https://togithub.com/yoniko) - build: fix MSVC warnings ([#​3495](https://togithub.com/facebook/zstd/issues/3495)) by [@​embg](https://togithub.com/embg) - doc: updated zstd specification to clarify corner cases, by [@​Cyan4973](https://togithub.com/Cyan4973) - doc: document how to create fat binaries for macos ([#​3568](https://togithub.com/facebook/zstd/issues/3568)) by [@​rickmark](https://togithub.com/rickmark) - misc: improve seekable format ingestion speed (~+100%) for very small chunk sizes ([#​3544](https://togithub.com/facebook/zstd/issues/3544)) by [@​Cyan4973](https://togithub.com/Cyan4973) - misc: `tests/fullbench` can benchmark multiple files ([#​3516](https://togithub.com/facebook/zstd/issues/3516)) by [@​dloidolt](https://togithub.com/dloidolt) #### Full change list (auto-generated) - Fix all MSVC warnings by [@​embg](https://togithub.com/embg) in [https://github.com/facebook/zstd/pull/3495](https://togithub.com/facebook/zstd/pull/3495) - Fix zstd-dll build missing dependencies by [@​yoniko](https://togithub.com/yoniko) in [https://github.com/facebook/zstd/pull/3496](https://togithub.com/facebook/zstd/pull/3496) - Bump github/codeql-action from 2.2.1 to 2.2.4 by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/facebook/zstd/pull/3503](https://togithub.com/facebook/zstd/pull/3503) - Github Action to generate Win64 artifacts by [@​Cyan4973](https://togithub.com/Cyan4973) in [https://github.com/facebook/zstd/pull/3491](https://togithub.com/facebook/zstd/pull/3491) - Use correct types in LZMA comp/decomp by [@​Hello71](https://togithub.com/Hello71) in [https://github.com/facebook/zstd/pull/3497](https://togithub.com/facebook/zstd/pull/3497) - Make Github workflows permissions read-only by default by [@​yoniko](https://togithub.com/yoniko) in [https://github.com/facebook/zstd/pull/3488](https://togithub.com/facebook/zstd/pull/3488) - CI Workflow for external compressors dependencies by [@​yoniko](https://togithub.com/yoniko) in [https://github.com/facebook/zstd/pull/3505](https://togithub.com/facebook/zstd/pull/3505) - Fix cli-tests issues by [@​daniellerozenblit](https://togithub.com/daniellerozenblit) in [https://github.com/facebook/zstd/pull/3509](https://togithub.com/facebook/zstd/pull/3509) - Fix Permissions on Publish Release Artifacts Job by [@​felixhandte](https://togithub.com/felixhandte) in [https://github.com/facebook/zstd/pull/3511](https://togithub.com/facebook/zstd/pull/3511) - Use `f`-variants of `chmod()` and `chown()` by [@​felixhandte](https://togithub.com/felixhandte) in [https://github.com/facebook/zstd/pull/3479](https://togithub.com/facebook/zstd/pull/3479) - Don't require CMake 3.18 or later by [@​kou](https://togithub.com/kou) in [https://github.com/facebook/zstd/pull/3510](https://togithub.com/facebook/zstd/pull/3510) - meson: always build the zstd binary when tests are enabled by [@​eli-schwartz](https://togithub.com/eli-schwartz) in [https://github.com/facebook/zstd/pull/3490](https://togithub.com/facebook/zstd/pull/3490) - \[bug-fix] Fix rare corruption bug affecting the block splitter by [@​terrelln](https://togithub.com/terrelln) in [https://github.com/facebook/zstd/pull/3517](https://togithub.com/facebook/zstd/pull/3517) - Clarify zstd specification for Huffman blocks by [@​Cyan4973](https://togithub.com/Cyan4973) in [https://github.com/facebook/zstd/pull/3514](https://togithub.com/facebook/zstd/pull/3514) - Fix typos found by codespell by [@​DimitriPapadopoulos](https://togithub.com/DimitriPapadopoulos) in [https://github.com/facebook/zstd/pull/3513](https://togithub.com/facebook/zstd/pull/3513) - Bump github/codeql-action from 2.2.4 to 2.2.5 by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/facebook/zstd/pull/3518](https://togithub.com/facebook/zstd/pull/3518) - fullbench with two files by [@​dloidolt](https://togithub.com/dloidolt) in [https://github.com/facebook/zstd/pull/3516](https://togithub.com/facebook/zstd/pull/3516) - Add initialization of clevel to static cdict ([#​3525](https://togithub.com/facebook/zstd/issues/3525)) by [@​yoniko](https://togithub.com/yoniko) in [https://github.com/facebook/zstd/pull/3527](https://togithub.com/facebook/zstd/pull/3527) - \[linux-kernel] Fix assert definition by [@​terrelln](https://togithub.com/terrelln) in [https://github.com/facebook/zstd/pull/3532](https://togithub.com/facebook/zstd/pull/3532) - Add ZSTD_set{C,F,}Params() helper functions by [@​terrelln](https://togithub.com/terrelln) in [https://github.com/facebook/zstd/pull/3530](https://togithub.com/facebook/zstd/pull/3530) - Clarify dstCapacity requirements by [@​terrelln](https://togithub.com/terrelln) in [https://github.com/facebook/zstd/pull/3531](https://togithub.com/facebook/zstd/pull/3531) - Mmap large dictionaries in patch-from mode by [@​daniellerozenblit](https://togithub.com/daniellerozenblit) in [https://github.com/facebook/zstd/pull/3486](https://togithub.com/facebook/zstd/pull/3486) - added clarifications for sizes of compressed huffman blocks and streams. by [@​Cyan4973](https://togithub.com/Cyan4973) in [https://github.com/facebook/zstd/pull/3538](https://togithub.com/facebook/zstd/pull/3538) - Simplify benchmark unit invocation API from CLI by [@​Cyan4973](https://togithub.com/Cyan4973) in [https://github.com/facebook/zstd/pull/3526](https://togithub.com/facebook/zstd/pull/3526) - Avoid Segfault Caused by Calling `setvbuf()` on Null File Pointer by [@​felixhandte](https://togithub.com/felixhandte) in [https://github.com/facebook/zstd/pull/3541](https://togithub.com/facebook/zstd/pull/3541) - Pin Moar Action Dependencies by [@​felixhandte](https://togithub.com/felixhandte) in [https://github.com/facebook/zstd/pull/3542](https://togithub.com/facebook/zstd/pull/3542) - Improved seekable format ingestion speed for small frame size by [@​Cyan4973](https://togithub.com/Cyan4973) in [https://github.com/facebook/zstd/pull/3544](https://togithub.com/facebook/zstd/pull/3544) - Reduce RowHash's tag space size by x2 by [@​yoniko](https://togithub.com/yoniko) in [https://github.com/facebook/zstd/pull/3543](https://togithub.com/facebook/zstd/pull/3543) - \[Bugfix] row hash tries to match position 0 by [@​yoniko](https://togithub.com/yoniko) in [https://github.com/facebook/zstd/pull/3548](https://togithub.com/facebook/zstd/pull/3548) - Bump github/codeql-action from 2.2.5 to 2.2.6 by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/facebook/zstd/pull/3549](https://togithub.com/facebook/zstd/pull/3549) - Add init once memory ([#​3528](https://togithub.com/facebook/zstd/issues/3528)) by [@​yoniko](https://togithub.com/yoniko) in [https://github.com/facebook/zstd/pull/3529](https://togithub.com/facebook/zstd/pull/3529) - Introduce salt into row hash ([#​3528](https://togithub.com/facebook/zstd/issues/3528) part 2) by [@​yoniko](https://togithub.com/yoniko) in [https://github.com/facebook/zstd/pull/3533](https://togithub.com/facebook/zstd/pull/3533) - added documentation for the seekable format by [@​Cyan4973](https://togithub.com/Cyan4973) in [https://github.com/facebook/zstd/pull/3547](https://togithub.com/facebook/zstd/pull/3547) - patch-from speed optimization by [@​daniellerozenblit](https://togithub.com/daniellerozenblit) in [https://github.com/facebook/zstd/pull/3545](https://togithub.com/facebook/zstd/pull/3545) - Deprecated bufferless and block level APIs by [@​terrelln](https://togithub.com/terrelln) in [https://github.com/facebook/zstd/pull/3534](https://togithub.com/facebook/zstd/pull/3534) - added documentation for LDM + dictionary compatibility by [@​Cyan4973](https://togithub.com/Cyan4973) in [https://github.com/facebook/zstd/pull/3553](https://togithub.com/facebook/zstd/pull/3553) - Fix a bug in the CLI tests newline processing, then simplify it further by [@​ppentchev](https://togithub.com/ppentchev) in [https://github.com/facebook/zstd/pull/3559](https://togithub.com/facebook/zstd/pull/3559) - \[lazy] Skip over incompressible data by [@​terrelln](https://togithub.com/terrelln) in [https://github.com/facebook/zstd/pull/3552](https://togithub.com/facebook/zstd/pull/3552) - Fix patch-from speed optimization by [@​daniellerozenblit](https://togithub.com/daniellerozenblit) in [https://github.com/facebook/zstd/pull/3556](https://togithub.com/facebook/zstd/pull/3556) - Bump actions/checkout from 3.3.0 to 3.5.0 by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/facebook/zstd/pull/3572](https://togithub.com/facebook/zstd/pull/3572) - \[easy] minor doc update for --rsyncable by [@​Cyan4973](https://togithub.com/Cyan4973) in [https://github.com/facebook/zstd/pull/3570](https://togithub.com/facebook/zstd/pull/3570) - \[contrib/pzstd] Select `-std=c++11` When Default is Older by [@​felixhandte](https://togithub.com/felixhandte) in [https://github.com/facebook/zstd/pull/3574](https://togithub.com/facebook/zstd/pull/3574) - Add instructions for building Universal2 on macOS via CMake by [@​rickmark](https://togithub.com/rickmark) in [https://github.com/facebook/zstd/pull/3568](https://togithub.com/facebook/zstd/pull/3568) - Provide an interface for fuzzing sequence producer plugins by [@​embg](https://togithub.com/embg) in [https://github.com/facebook/zstd/pull/3551](https://togithub.com/facebook/zstd/pull/3551) - mmap for windows by [@​daniellerozenblit](https://togithub.com/daniellerozenblit) in [https://github.com/facebook/zstd/pull/3557](https://togithub.com/facebook/zstd/pull/3557) - Bump github/codeql-action from 2.2.6 to 2.2.8 by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/facebook/zstd/pull/3573](https://togithub.com/facebook/zstd/pull/3573) - Disable linker flag detection on MSVC/ClangCL. by [@​tru](https://togithub.com/tru) in [https://github.com/facebook/zstd/pull/3569](https://togithub.com/facebook/zstd/pull/3569) - Couple tweaks to improve decompression speed with clang PGO compilation by [@​zhuhan0](https://togithub.com/zhuhan0) in [https://github.com/facebook/zstd/pull/3576](https://togithub.com/facebook/zstd/pull/3576) - Increase tests timeout by [@​dvoropaev](https://togithub.com/dvoropaev) in [https://github.com/facebook/zstd/pull/3540](https://togithub.com/facebook/zstd/pull/3540) - added a Clang-CL Windows test to CI by [@​Cyan4973](https://togithub.com/Cyan4973) in [https://github.com/facebook/zstd/pull/3579](https://togithub.com/facebook/zstd/pull/3579) - Seekable format read optimization by [@​Cyan4973](https://togithub.com/Cyan4973) in [https://github.com/facebook/zstd/pull/3581](https://togithub.com/facebook/zstd/pull/3581) - Check that `dest` is valid for decompression by [@​daniellerozenblit](https://togithub.com/daniellerozenblit) in [https://github.com/facebook/zstd/pull/3555](https://togithub.com/facebook/zstd/pull/3555) - fix decompression with -o writing into a block device by [@​Cyan4973](https://togithub.com/Cyan4973) in [https://github.com/facebook/zstd/pull/3584](https://togithub.com/facebook/zstd/pull/3584) - updated version number to v1.5.5 by [@​Cyan4973](https://togithub.com/Cyan4973) in [https://github.com/facebook/zstd/pull/3577](https://togithub.com/facebook/zstd/pull/3577) #### New Contributors - [@​kou](https://togithub.com/kou) made their first contribution in [https://github.com/facebook/zstd/pull/3510](https://togithub.com/facebook/zstd/pull/3510) - [@​dloidolt](https://togithub.com/dloidolt) made their first contribution in [https://github.com/facebook/zstd/pull/3516](https://togithub.com/facebook/zstd/pull/3516) - [@​ppentchev](https://togithub.com/ppentchev) made their first contribution in [https://github.com/facebook/zstd/pull/3559](https://togithub.com/facebook/zstd/pull/3559) - [@​rickmark](https://togithub.com/rickmark) made their first contribution in [https://github.com/facebook/zstd/pull/3568](https://togithub.com/facebook/zstd/pull/3568) - [@​dvoropaev](https://togithub.com/dvoropaev) made their first contribution in [https://github.com/facebook/zstd/pull/3540](https://togithub.com/facebook/zstd/pull/3540) **Full Changelog**: https://github.com/facebook/zstd/compare/v1.5.4...v1.5.5 ### [`v1.5.4`](https://togithub.com/facebook/zstd/releases/tag/v1.5.4): Zstandard v1.5.4 [Compare Source](https://togithub.com/facebook/zstd/compare/v1.5.2...v1.5.4) Zstandard `v1.5.4` is a pretty big release benefiting from one year of work, spread over > 650 commits. It offers significant performance improvements across multiple scenarios, as well as new features (detailed below). There is a crop of little bug fixes too, a few ones targeting the 32-bit mode are important enough to make this release a recommended upgrade. ##### Various Speed improvements This release has accumulated a number of scenario-specific improvements, that cumulatively benefit a good portion of installed base in one way or another. Among the easier ones to describe, the repository has received several contributions for `arm` optimizations, notably from [@​JunHe77](https://togithub.com/JunHe77) and [@​danlark1](https://togithub.com/danlark1). And [@​terrelln](https://togithub.com/terrelln) has improved decompression speed for non-x64 systems, including `arm`. The combination of this work is visible in the following example, using an M1-Pro (`aarch64` architecture) : | cpu | function | corpus | `v1.5.2` | `v1.5.4` | Improvement | | --- | --- | --- | --- | --- | --- | | M1 Pro | decompress | `silesia.tar` | 1370 MB/s | 1480 MB/s | + 8% | | Galaxy S22 | decompress | `silesia.tar` | 1150 MB/s | 1200 MB/s | + 4% | Middle compression levels (5-12) receive some care too, with [@​terrelln](https://togithub.com/terrelln) improving the dispatch engine, and [@​danlark1](https://togithub.com/danlark1) offering `NEON` optimizations. Exact speed up vary depending on platform, cpu, compiler, and compression level, though one can expect gains ranging from +1 to +10% depending on scenarios. | cpu | function | corpus | `v1.5.2` | `v1.5.4` | Improvement | | --- | --- | --- | ---:| ---:| --- | | i7-9700k | compress -6 | `silesia.tar` | 110 MB/s | 121 MB/s | +10% | Galaxy S22 | compress -6 | `silesia.tar` | 98 MB/s | 103 MB/s | +5% | M1 Pro | compress -6 | `silesia.tar` | 122 MB/s | 130 MB/s | +6.5% | i7-9700k | compress -9 | `silesia.tar` | 64 MB/s | 70 MB/s | +9.5% | Galaxy S22 | compress -9 | `silesia.tar` | 51 MB/s | 52 MB/s | +1% | M1 Pro | compress -9 | `silesia.tar` | 77 MB/s | 86 MB/s | +11.5% | i7-9700k | compress -12 | `silesia.tar` | 31.6 MB/s | 31.8 MB/s | +0.5% | Galaxy S22 | compress -12 | `silesia.tar` | 20.9 MB/s | 22.1 MB/s | +5% | M1 Pro | compress -12 | `silesia.tar` | 36.1 MB/s | 39.7 MB/s | +10% Speed of the streaming compression interface has been improved by [@​embg](https://togithub.com/embg) in scenarios involving large files (where size is a multiple of the `windowSize` parameter). The improvement is mostly perceptible at high speeds (i.e. ~level 1). In the following sample, the measurement is taken directly at `ZSTD_compressStream()` function call, using a dedicated benchmark tool `tests/fullbench`. | cpu | function | corpus | `v1.5.2` | `v1.5.4` | Improvement | | --- | --- | --- | --- | --- | --- | | i7-9700k | `ZSTD_compressStream()` -1 | `silesia.tar` | 392 MB/s | 429 MB/s | +9.5% | | Galaxy S22 | `ZSTD_compressStream()` -1 | `silesia.tar` | 380 MB/s | 430 MB/s | +13% | | M1 Pro | `ZSTD_compressStream()` -1 | `silesia.tar` | 476 MB/s | 539 MB/s | +13% | Finally, dictionary compression speed has received a good boost by [@​embg](https://togithub.com/embg). Exact outcome varies depending on system and corpus. The following result is achieved by cutting the `enwik8` compression corpus into 1KB blocks, generating a dictionary from these blocks, and then benchmarking the compression speed at level 1. | cpu | function | corpus | `v1.5.2` | `v1.5.4` | Improvement | | --- | --- | --- | --- | --- | --- | | i7-9700k | dictionary compress | `enwik8` -B1K | 125 MB/s | 165 MB/s | +32% | | Galaxy S22 | dictionary compress | `enwik8` -B1K | 138 MB/s | 166 MB/s | +20% | | M1 Pro | dictionary compress | `enwik8` -B1K | 155 MB/s | 195 MB/s | +25 % | There are a few more scenario-specifics improvements listed in the `changelog` section below. ##### I/O Performance improvements The 1.5.4 release improves IO performance of `zstd` CLI, by using system buffers (`macos`) and adding a new asynchronous I/O capability, enabled by default on large files (when threading is available). The user can also explicitly control this capability with the `--[no-]asyncio` flag . These new threads remove the need to block on IO operations. The impact is mostly noticeable when decompressing large files (>= a few MBs), though exact outcome depends on environment and run conditions. Decompression speed gets significant gains due to its single-threaded serial nature and the high speeds involved. In some cases we observe up to double performance improvement (local Mac machines) and a wide +15-45% benefit on Intel Linux servers (see table for details).\ On the compression side of things, we’ve measured up to 5% improvements. The impact is lower because compression is already partially asynchronous via the internal MT mode (see release [v1.3.4](https://togithub.com/facebook/zstd/releases/tag/v1.3.4)). The following table shows the elapsed run time for decompressions of `silesia` and `enwik8` on several platforms - some Skylake-era Linux servers and an M1 MacbookPro. It compares the time it takes for version `v1.5.2` to version `v1.5.4` with asyncio on and off. platform | corpus | `v1.5.2` | `v1.5.4-no-asyncio` | `v1.5.4` | Improvement \-- | -- | -- | -- | -- | -- Xeon D-2191A CentOS8 | `enwik8` | 280 MB/s | 280 MB/s | 324 MB/s | +16% Xeon D-2191A CentOS8 | `silesia.tar` | 303 MB/s | 302 MB/s | 386 MB/s | +27% i7-1165g7 win10 | `enwik8` | 270 MB/s | 280 MB/s | 350 MB/s | +27% i7-1165g7 win10 | `silesia.tar` | 450 MB/s | 440 MB/s | 580 MB/s | +28% i7-9700K Ubuntu20 | `enwik8` | 600 MB/s | 604 MB/s | 829 MB/s | +38% i7-9700K Ubuntu20 | `silesia.tar` | 683 MB/s | 678 MB/s | 991 MB/s | +45% Galaxy S22 | `enwik8` | 360 MB/s | 420 MB/s | 515 MB/s | +70% Galaxy S22 | `silesia.tar` | 310 MB/s | 320 MB/s | 580 MB/s | +85% MBP M1 | `enwik8` | 428 MB/s | 734 MB/s | 815 MB/s | +90% MBP M1 | `silesia.tar` | 465 MB/s | 875 MB/s | 1001 MB/s | +115% ##### Support of externally-defined sequence producers `libzstd` can now support external sequence producers via a new advanced registration function `ZSTD_registerSequenceProducer()` ([#​3333](https://togithub.com/facebook/zstd/issues/3333)). This API allows users to provide their own custom sequence producer which libzstd invokes to process each block. The produced list of sequences (literals and matches) is then post-processed by libzstd to produce valid compressed blocks. This block-level offload API is a more granular complement of the existing frame-level offload API `compressSequences()` (introduced in `v1.5.1`). It offers an easier migration story for applications already integrated with `libzstd`: the user application continues to invoke the same compression functions `ZSTD_compress2()` or `ZSTD_compressStream2()` as usual, and transparently benefits from the specific properties of the external sequence producer. For example, the sequence producer could be tuned to take advantage of known characteristics of the input, to offer better speed / ratio. One scenario that becomes possible is to combine this capability with hardware-accelerated matchfinders, such as the Intel® QuickAssist accelerator (Intel® QAT) provided in server CPUs such as the 4th Gen Intel® Xeon® Scalable processors (previously codenamed Sapphire Rapids). More details to be provided in future communications. #### Change Log perf: +20% faster huffman decompression for targets that can't compile x64 assembly ([#​3449](https://togithub.com/facebook/zstd/issues/3449), [@​terrelln](https://togithub.com/terrelln)) perf: up to +10% faster streaming compression at levels 1-2 ([#​3114](https://togithub.com/facebook/zstd/issues/3114), [@​embg](https://togithub.com/embg)) perf: +4-13% for levels 5-12 by optimizing function generation ([#​3295](https://togithub.com/facebook/zstd/issues/3295), [@​terrelln](https://togithub.com/terrelln)) pref: +3-11% compression speed for `arm` target ([#​3199](https://togithub.com/facebook/zstd/issues/3199), [#​3164](https://togithub.com/facebook/zstd/issues/3164), [#​3145](https://togithub.com/facebook/zstd/issues/3145), [#​3141](https://togithub.com/facebook/zstd/issues/3141), [#​3138](https://togithub.com/facebook/zstd/issues/3138), [@​JunHe77](https://togithub.com/JunHe77) and [#​3139](https://togithub.com/facebook/zstd/issues/3139), [#​3160](https://togithub.com/facebook/zstd/issues/3160), [@​danlark1](https://togithub.com/danlark1)) perf: +5-30% faster dictionary compression at levels 1-4 ([#​3086](https://togithub.com/facebook/zstd/issues/3086), [#​3114](https://togithub.com/facebook/zstd/issues/3114), [#​3152](https://togithub.com/facebook/zstd/issues/3152), [@​embg](https://togithub.com/embg)) perf: +10-20% cold dict compression speed by prefetching CDict tables ([#​3177](https://togithub.com/facebook/zstd/issues/3177), [@​embg](https://togithub.com/embg)) perf: +1% faster compression by removing a branch in ZSTD_fast_noDict ([#​3129](https://togithub.com/facebook/zstd/issues/3129), [@​felixhandte](https://togithub.com/felixhandte)) perf: Small compression ratio improvements in high compression mode ([#​2983](https://togithub.com/facebook/zstd/issues/2983), [#​3391](https://togithub.com/facebook/zstd/issues/3391), [@​Cyan4973](https://togithub.com/Cyan4973) and [#​3285](https://togithub.com/facebook/zstd/issues/3285), [#​3302](https://togithub.com/facebook/zstd/issues/3302), [@​daniellerozenblit](https://togithub.com/daniellerozenblit)) perf: small speed improvement by better detecting `STATIC_BMI2` for `clang` ([#​3080](https://togithub.com/facebook/zstd/issues/3080), [@​TocarIP](https://togithub.com/TocarIP)) perf: Improved streaming performance when `ZSTD_c_stableInBuffer` is set ([#​2974](https://togithub.com/facebook/zstd/issues/2974), [@​Cyan4973](https://togithub.com/Cyan4973)) cli: Asynchronous I/O for improved cli speed ([#​2975](https://togithub.com/facebook/zstd/issues/2975), [#​2985](https://togithub.com/facebook/zstd/issues/2985), [#​3021](https://togithub.com/facebook/zstd/issues/3021), [#​3022](https://togithub.com/facebook/zstd/issues/3022), [@​yoniko](https://togithub.com/yoniko)) cli: Change `zstdless` behavior to align with `zless` ([#​2909](https://togithub.com/facebook/zstd/issues/2909), [@​binhdvo](https://togithub.com/binhdvo)) cli: Keep original file if `-c` or `--stdout` is given ([#​3052](https://togithub.com/facebook/zstd/issues/3052), [@​dirkmueller](https://togithub.com/dirkmueller)) cli: Keep original files when result is concatenated into a single output with `-o` ([#​3450](https://togithub.com/facebook/zstd/issues/3450), [@​Cyan4973](https://togithub.com/Cyan4973)) cli: Preserve Permissions and Ownership of regular files ([#​3432](https://togithub.com/facebook/zstd/issues/3432), [@​felixhandte](https://togithub.com/felixhandte)) cli: Print zlib/lz4/lzma library versions with `-vv` ([#​3030](https://togithub.com/facebook/zstd/issues/3030), [@​terrelln](https://togithub.com/terrelln)) cli: Print checksum value for single frame files with `-lv` ([#​3332](https://togithub.com/facebook/zstd/issues/3332), [@​Cyan4973](https://togithub.com/Cyan4973)) cli: Print `dictID` when present with `-lv` ([#​3184](https://togithub.com/facebook/zstd/issues/3184), [@​htnhan](https://togithub.com/htnhan)) cli: when `stderr` is *not* the console, disable status updates, but preserve final summary ([#​3458](https://togithub.com/facebook/zstd/issues/3458), [@​Cyan4973](https://togithub.com/Cyan4973)) cli: support `--best` and `--no-name` in `gzip` compatibility mode ([#​3059](https://togithub.com/facebook/zstd/issues/3059), [@​dirkmueller](https://togithub.com/dirkmueller)) cli: support for `posix` high resolution timer `clock_gettime()`, for improved benchmark accuracy ([#​3423](https://togithub.com/facebook/zstd/issues/3423), [@​Cyan4973](https://togithub.com/Cyan4973)) cli: improved help/usage (`-h`, `-H`) formatting ([#​3094](https://togithub.com/facebook/zstd/issues/3094), [@​dirkmueller](https://togithub.com/dirkmueller) and [#​3385](https://togithub.com/facebook/zstd/issues/3385), [@​jonpalmisc](https://togithub.com/jonpalmisc)) cli: Fix better handling of bogus numeric values ([#​3268](https://togithub.com/facebook/zstd/issues/3268), [@​ctkhanhly](https://togithub.com/ctkhanhly)) cli: Fix input consists of multiple files *and* `stdin` ([#​3222](https://togithub.com/facebook/zstd/issues/3222), [@​yoniko](https://togithub.com/yoniko)) cli: Fix tiny files passthrough ([#​3215](https://togithub.com/facebook/zstd/issues/3215), [@​cgbur](https://togithub.com/cgbur)) cli: Fix for `-r` on empty directory ([#​3027](https://togithub.com/facebook/zstd/issues/3027), [@​brailovich](https://togithub.com/brailovich)) cli: Fix empty string as argument for `--output-dir-*` ([#​3220](https://togithub.com/facebook/zstd/issues/3220), [@​embg](https://togithub.com/embg)) cli: Fix decompression memory usage reported by `-vv --long` ([#​3042](https://togithub.com/facebook/zstd/issues/3042), [@​u1f35c](https://togithub.com/u1f35c), and [#​3232](https://togithub.com/facebook/zstd/issues/3232), [@​zengyijing](https://togithub.com/zengyijing)) cli: Fix infinite loop when empty input is passed to trainer ([#​3081](https://togithub.com/facebook/zstd/issues/3081), [@​terrelln](https://togithub.com/terrelln)) cli: Fix `--adapt` doesn't work when `--no-progress` is also set ([#​3354](https://togithub.com/facebook/zstd/issues/3354), [@​terrelln](https://togithub.com/terrelln)) api: Support for External Sequence Producer ([#​3333](https://togithub.com/facebook/zstd/issues/3333), [@​embg](https://togithub.com/embg)) api: Support for in-place decompression ([#​3432](https://togithub.com/facebook/zstd/issues/3432), [@​terrelln](https://togithub.com/terrelln)) api: New `ZSTD_CCtx_setCParams()` function, set all parameters defined in a `ZSTD_compressionParameters` structure ([#​3403](https://togithub.com/facebook/zstd/issues/3403), [@​Cyan4973](https://togithub.com/Cyan4973)) api: Streaming decompression detects incorrect header ID sooner ([#​3175](https://togithub.com/facebook/zstd/issues/3175), [@​Cyan4973](https://togithub.com/Cyan4973)) api: Window size resizing optimization for edge case ([#​3345](https://togithub.com/facebook/zstd/issues/3345), [@​daniellerozenblit](https://togithub.com/daniellerozenblit)) api: More accurate error codes for busy-loop scenarios ([#​3413](https://togithub.com/facebook/zstd/issues/3413), [#​3455](https://togithub.com/facebook/zstd/issues/3455), [@​Cyan4973](https://togithub.com/Cyan4973)) api: Fix limit overflow in `compressBound` and `decompressBound` ([#​3362](https://togithub.com/facebook/zstd/issues/3362), [#​3373](https://togithub.com/facebook/zstd/issues/3373), Cyan4973) reported by [@​nigeltao](https://togithub.com/nigeltao) api: Deprecate several advanced experimental functions: streaming ([#​3408](https://togithub.com/facebook/zstd/issues/3408), [@​embg](https://togithub.com/embg)), copy ([#​3196](https://togithub.com/facebook/zstd/issues/3196), [@​mileshu](https://togithub.com/mileshu)) bug: Fix corruption that rarely occurs in 32-bit mode with wlog=25 ([#​3361](https://togithub.com/facebook/zstd/issues/3361), [@​terrelln](https://togithub.com/terrelln)) bug: Fix for block-splitter ([#​3033](https://togithub.com/facebook/zstd/issues/3033), [@​Cyan4973](https://togithub.com/Cyan4973)) bug: Fixes for Sequence Compression API ([#​3023](https://togithub.com/facebook/zstd/issues/3023), [#​3040](https://togithub.com/facebook/zstd/issues/3040), [@​Cyan4973](https://togithub.com/Cyan4973)) bug: Fix leaking thread handles on Windows ([#​3147](https://togithub.com/facebook/zstd/issues/3147), [@​animalize](https://togithub.com/animalize)) bug: Fix timing issues with cmake/meson builds ([#​3166](https://togithub.com/facebook/zstd/issues/3166), [#​3167](https://togithub.com/facebook/zstd/issues/3167), [#​3170](https://togithub.com/facebook/zstd/issues/3170), [@​Cyan4973](https://togithub.com/Cyan4973)) build: Allow user to select legacy level for cmake ([#​3050](https://togithub.com/facebook/zstd/issues/3050), [@​shadchin](https://togithub.com/shadchin)) build: Enable legacy support by default in cmake ([#​3079](https://togithub.com/facebook/zstd/issues/3079), [@​niamster](https://togithub.com/niamster)) build: Meson build script improvements ([#​3039](https://togithub.com/facebook/zstd/issues/3039), [#​3120](https://togithub.com/facebook/zstd/issues/3120), [#​3122](https://togithub.com/facebook/zstd/issues/3122), [#​3327](https://togithub.com/facebook/zstd/issues/3327), [#​3357](https://togithub.com/facebook/zstd/issues/3357), [@​eli-schwartz](https://togithub.com/eli-schwartz) and [#​3276](https://togithub.com/facebook/zstd/issues/3276), [@​neheb](https://togithub.com/neheb)) build: Add aarch64 to supported architectures for zstd_trace ([#​3054](https://togithub.com/facebook/zstd/issues/3054), [@​ooosssososos](https://togithub.com/ooosssososos)) build: support AIX architecture ([#​3219](https://togithub.com/facebook/zstd/issues/3219), [@​qiongsiwu](https://togithub.com/qiongsiwu)) build: Fix `ZSTD_LIB_MINIFY` build macro, which now reduces static library size by half ([#​3366](https://togithub.com/facebook/zstd/issues/3366), [@​terrelln](https://togithub.com/terrelln)) build: Fix Windows issues with Multithreading translation layer ([#​3364](https://togithub.com/facebook/zstd/issues/3364), [#​3380](https://togithub.com/facebook/zstd/issues/3380), [@​yoniko](https://togithub.com/yoniko)) and ARM64 target ([#​3320](https://togithub.com/facebook/zstd/issues/3320), [@​cwoffenden](https://togithub.com/cwoffenden)) build: Fix `cmake` script ([#​3382](https://togithub.com/facebook/zstd/issues/3382), [#​3392](https://togithub.com/facebook/zstd/issues/3392), [@​terrelln](https://togithub.com/terrelln) and [#​3252](https://togithub.com/facebook/zstd/issues/3252) [@​Tachi107](https://togithub.com/Tachi107) and [#​3167](https://togithub.com/facebook/zstd/issues/3167) [@​Cyan4973](https://togithub.com/Cyan4973)) doc: Updated man page, providing more details for `--train` mode ([#​3112](https://togithub.com/facebook/zstd/issues/3112), [@​Cyan4973](https://togithub.com/Cyan4973)) doc: Add decompressor errata document ([#​3092](https://togithub.com/facebook/zstd/issues/3092), [@​terrelln](https://togithub.com/terrelln)) misc: Enable Intel CET ([#​2992](https://togithub.com/facebook/zstd/issues/2992), [#​2994](https://togithub.com/facebook/zstd/issues/2994), [@​hjl-tools](https://togithub.com/hjl-tools)) misc: Fix `contrib/` seekable format ([#​3058](https://togithub.com/facebook/zstd/issues/3058), [@​yhoogstrate](https://togithub.com/yhoogstrate) and [#​3346](https://togithub.com/facebook/zstd/issues/3346), [@​daniellerozenblit](https://togithub.com/daniellerozenblit)) misc: Improve speed of the one-file library generator ([#​3241](https://togithub.com/facebook/zstd/issues/3241), [@​wahern](https://togithub.com/wahern) and [#​3005](https://togithub.com/facebook/zstd/issues/3005), [@​cwoffenden](https://togithub.com/cwoffenden)) #### PR list (generated by Github) - x86-64: Enable Intel CET by [@​hjl-tools](https://togithub.com/hjl-tools) in [https://github.com/facebook/zstd/pull/2992](https://togithub.com/facebook/zstd/pull/2992) - Add GitHub Action Checking that Zstd Runs Successfully Under CET by [@​felixhandte](https://togithub.com/felixhandte) in [https://github.com/facebook/zstd/pull/3015](https://togithub.com/facebook/zstd/pull/3015) - \[opt] minor compression ratio improvement by [@​Cyan4973](https://togithub.com/Cyan4973) in [https://github.com/facebook/zstd/pull/2983](https://togithub.com/facebook/zstd/pull/2983) - Simplify HUF_decompress4X2\_usingDTable_internal_bmi2\_asm_loop by [@​WojciechMula](https://togithub.com/WojciechMula) in [https://github.com/facebook/zstd/pull/3013](https://togithub.com/facebook/zstd/pull/3013) - Async write for decompression by [@​yoniko](https://togithub.com/yoniko) in [https://github.com/facebook/zstd/pull/2975](https://togithub.com/facebook/zstd/pull/2975) - ZSTD CLI: Use buffered output by [@​yoniko](https://togithub.com/yoniko) in [https://github.com/facebook/zstd/pull/2985](https://togithub.com/facebook/zstd/pull/2985) - Use faster Python script to amalgamate by [@​cwoffenden](https://togithub.com/cwoffenden) in [https://github.com/facebook/zstd/pull/3005](https://togithub.com/facebook/zstd/pull/3005) - Change zstdless behavior to align with zless by [@​binhdvo](https://togithub.com/binhdvo) in [https://github.com/facebook/zstd/pull/2909](https://togithub.com/facebook/zstd/pull/2909) - AsyncIO compression part 1 - refactor of existing asyncio code by [@​yoniko](https://togithub.com/yoniko) in [https://github.com/facebook/zstd/pull/3021](https://togithub.com/facebook/zstd/pull/3021) - Converge sumtype (offset | repcode) numeric representation towards offBase by [@​Cyan4973](https://togithub.com/Cyan4973) in [https://github.com/facebook/zstd/pull/2965](https://togithub.com/facebook/zstd/pull/2965) - fix sequence compression API in Explicit Delimiter mode by [@​Cyan4973](https://togithub.com/Cyan4973) in [https://github.com/facebook/zstd/pull/3023](https://togithub.com/facebook/zstd/pull/3023) - Lazy parameters adaptation (part 1 - ZSTD_c_stableInBuffer) by [@​Cyan4973](https://togithub.com/Cyan4973) in [https://github.com/facebook/zstd/pull/2974](https://togithub.com/facebook/zstd/pull/2974) - Print zlib/lz4/lzma library versions in verbose version output by [@​terrelln](https://togithub.com/terrelln) in [https://github.com/facebook/zstd/pull/3030](https://togithub.com/facebook/zstd/pull/3030) - fix for -r on empty directory by [@​brailovich](https://togithub.com/brailovich) in [https://github.com/facebook/zstd/pull/3027](https://togithub.com/facebook/zstd/pull/3027) - Add new CLI testing platform by [@​terrelln](https://togithub.com/terrelln) in [https://github.com/facebook/zstd/pull/3020](https://togithub.com/facebook/zstd/pull/3020) - AsyncIO compression part 2 - added async read and asyncio to compression code by [@​yoniko](https://togithub.com/yoniko) in [https://github.com/facebook/zstd/pull/3022](https://togithub.com/facebook/zstd/pull/3022) - Macos playtest envvars fix by [@​yoniko](https://togithub.com/yoniko) in [https://github.com/facebook/zstd/pull/3035](https://togithub.com/facebook/zstd/pull/3035) - Fix required decompression memory usage reported by -vv + --long by [@​u1f35c](https://togithub.com/u1f35c) in [https://github.com/facebook/zstd/pull/3042](https://togithub.com/facebook/zstd/pull/3042) - Select legacy level for cmake by [@​shadchin](https://togithub.com/shadchin) in [https://github.com/facebook/zstd/pull/3050](https://togithub.com/facebook/zstd/pull/3050) - \[trace] Add aarch64 to supported architectures for zstd_trace by [@​ooosssososos](https://togithub.com/ooosssososos) in [https://github.com/facebook/zstd/pull/3054](https://togithub.com/facebook/zstd/pull/3054) - New features for largeNbDicts benchmark by [@​embg](https://togithub.com/embg) in [https://github.com/facebook/zstd/pull/3063](https://togithub.com/facebook/zstd/pull/3063) - Use helper function for bit manipulations. by [@​TocarIP](https://togithub.com/TocarIP) in [https://github.com/facebook/zstd/pull/3075](https://togithub.com/facebook/zstd/pull/3075) - \[programs] Fix infinite loop when empty input is passed to trainer by [@​terrelln](https://togithub.com/terrelln) in [https://github.com/facebook/zstd/pull/3081](https://togithub.com/facebook/zstd/pull/3081) - Enable STATIC_BMI2 for gcc/clang by [@​TocarIP](https://togithub.com/TocarIP) in [https://github.com/facebook/zstd/pull/3080](https://togithub.com/facebook/zstd/pull/3080) - build:cmake: enable ZSTD legacy support by default by [@​niamster](https://togithub.com/niamster) in [https://github.com/facebook/zstd/pull/3079](https://togithub.com/facebook/zstd/pull/3079) - Implement more gzip compatibility ([#​3037](https://togithub.com/facebook/zstd/issues/3037)) by [@​dirkmueller](https://togithub.com/dirkmueller) in [https://github.com/facebook/zstd/pull/3059](https://togithub.com/facebook/zstd/pull/3059) - \[doc] Add decompressor errata document by [@​terrelln](https://togithub.com/terrelln) in [https://github.com/facebook/zstd/pull/3092](https://togithub.com/facebook/zstd/pull/3092) - Handle newer less versions in zstdless testing by [@​dirkmueller](https://togithub.com/dirkmueller) in [https://github.com/facebook/zstd/pull/3093](https://togithub.com/facebook/zstd/pull/3093) - \[contrib]\[linux] Fix a warning in zstd_reset_cstream() by [@​cyberknight777](https://togithub.com/cyberknight777) in [https://github.com/facebook/zstd/pull/3088](https://togithub.com/facebook/zstd/pull/3088) - Software pipeline for ZSTD_compressBlock_fast_dictMatchState (+5-6% compression speed) by [@​embg](https://togithub.com/embg) in [https://github.com/facebook/zstd/pull/3086](https://togithub.com/facebook/zstd/pull/3086) - Keep original file if -c or --stdout is given by [@​dirkmueller](https://togithub.com/dirkmueller) in [https://github.com/facebook/zstd/pull/3052](https://togithub.com/facebook/zstd/pull/3052) - Split help in long and short version, cleanup formatting by [@​dirkmueller](https://togithub.com/dirkmueller) in [https://github.com/facebook/zstd/pull/3094](https://togithub.com/facebook/zstd/pull/3094) - updated man page, providing more details for --train mode by [@​Cyan4973](https://togithub.com/Cyan4973) in [https://github.com/facebook/zstd/pull/3112](https://togithub.com/facebook/zstd/pull/3112) - Meson fixups for Windows by [@​eli-schwartz](https://togithub.com/eli-schwartz) in [https://github.com/facebook/zstd/pull/3039](https://togithub.com/facebook/zstd/pull/3039) - meson: for internal linkage, link to both libzstd and a static copy of it by [@​eli-schwartz](https://togithub.com/eli-schwartz) in [https://github.com/facebook/zstd/pull/3122](https://togithub.com/facebook/zstd/pull/3122) - Software pipeline for ZSTD_compressBlock_fast_extDict (+4-9% compression speed) by [@​embg](https://togithub.com/embg) in [https://github.com/facebook/zstd/pull/3114](https://togithub.com/facebook/zstd/pull/3114) - ZSTD_fast_noDict: Avoid Safety Check When Writing `ip1` into Table by [@​felixhandte](https://togithub.com/felixhandte) in [https://github.com/facebook/zstd/pull/3129](https://togithub.com/facebook/zstd/pull/3129) - Correct and clarify repcode offset history logic by [@​embg](https://togithub.com/embg) in [https://github.com/facebook/zstd/pull/3127](https://togithub.com/facebook/zstd/pull/3127) - \[lazy] Optimize ZSTD_row_getMatchMask for levels 8-10 for ARM by [@​danlark1](https://togithub.com/danlark1) in [https://github.com/facebook/zstd/pull/3139](https://togithub.com/facebook/zstd/pull/3139) - fix leaking thread handles on Windows by [@​animalize](https://togithub.com/animalize) in [https://github.com/facebook/zstd/pull/3147](https://togithub.com/facebook/zstd/pull/3147) - Remove expensive assert in --rsyncable hot loop by [@​terrelln](https://togithub.com/terrelln) in [https://github.com/facebook/zstd/pull/3154](https://togithub.com/facebook/zstd/pull/3154) - Bugfix for huge dictionaries by [@​embg](https://togithub.com/embg) in [https://github.com/facebook/zstd/pull/3157](https://togithub.com/facebook/zstd/pull/3157) - common: apply two stage copy to aarch64 by [@​JunHe77](https://togithub.com/JunHe77) in [https://github.com/facebook/zstd/pull/3145](https://togithub.com/facebook/zstd/pull/3145) - dec: adjust seqSymbol load on aarch64 by [@​JunHe77](https://togithub.com/JunHe77) in [https://github.com/facebook/zstd/pull/3141](https://togithub.com/facebook/zstd/pull/3141) - Fix big endian ARM NEON path by [@​danlark1](https://togithub.com/danlark1) in [https://github.com/facebook/zstd/pull/3160](https://togithub.com/facebook/zstd/pull/3160) - \[contrib] largeNbDicts bugfix + improvements by [@​embg](https://togithub.com/embg) in [https://github.com/facebook/zstd/pull/3161](https://togithub.com/facebook/zstd/pull/3161) - display a warning message when using C90 clock_t by [@​Cyan4973](https://togithub.com/Cyan4973) in [https://github.com/facebook/zstd/pull/3166](https://togithub.com/facebook/zstd/pull/3166) - remove explicit standard setting from cmake script by [@​Cyan4973](https://togithub.com/Cyan4973) in [https://github.com/facebook/zstd/pull/3167](https://togithub.com/facebook/zstd/pull/3167) - removed gnu99 statement from meson recipe by [@​Cyan4973](https://togithub.com/Cyan4973) in [https://github.com/facebook/zstd/pull/3170](https://togithub.com/facebook/zstd/pull/3170) - "Short cache" optimization for level 1-4 DMS (+5-30% compression speed) by [@​embg](https://togithub.com/embg) in [https://github.com/facebook/zstd/pull/3152](https://togithub.com/facebook/zstd/pull/3152) - Streaming decompression can detect incorrect header ID sooner by [@​Cyan4973](https://togithub.com/Cyan4973) in [https://github.com/facebook/zstd/pull/3175](https://togithub.com/facebook/zstd/pull/3175) - Add prefetchCDictTables CCtxParam (+10-20% cold dict compression speed) by [@​embg](https://togithub.com/embg) in [https://github.com/facebook/zstd/pull/3177](https://togithub.com/facebook/zstd/pull/3177) - Fix ZSTD_BUILD_TESTS=ON with MSVC by [@​nocnokneo](https://togithub.com/nocnokneo) in [https://github.com/facebook/zstd/pull/3180](https://togithub.com/facebook/zstd/pull/3180) - zstd -lv to show dictID by [@​htnhan](https://togithub.com/htnhan) in [https://github.com/facebook/zstd/pull/3184](https://togithub.com/facebook/zstd/pull/3184) - Intial commit to address 3090. Added support to decompress empty block. by [@​udayanbapat](https://togithub.com/udayanbapat) in [https://github.com/facebook/zstd/pull/3118](https://togithub.com/facebook/zstd/pull/3118) - \[largeNbDicts] Second try at fixing decompression segfault to always create compressInstructions by [@​zhuhan0](https://togithub.com/zhuhan0) in [https://github.com/facebook/zstd/pull/3209](https://togithub.com/facebook/zstd/pull/3209) - Clarify benchmark chunking docstring by [@​embg](https://togithub.com/embg) in [https://github.com/facebook/zstd/pull/3197](https://togithub.com/facebook/zstd/pull/3197) - decomp: add prefetch for matched seq on aarch64 by [@​JunHe77](https://togithub.com/JunHe77) in [https://github.com/facebook/zstd/pull/3164](https://togithub.com/facebook/zstd/pull/3164) - lib: add hint to generate more pipeline friendly code by [@​JunHe77](https://togithub.com/JunHe77) in [https://github.com/facebook/zstd/pull/3138](https://togithub.com/facebook/zstd/pull/3138) - \[AIX] Fix Compiler Flags and Bugs on AIX to Pass All Tests by [@​qiongsiwu](https://togithub.com/qiongsiwu) in [https://github.com/facebook/zstd/pull/3219](https://togithub.com/facebook/zstd/pull/3219) - zlibWrapper: Update for zlib 1.2.12 by [@​orbea](https://togithub.com/orbea) in [https://github.com/facebook/zstd/pull/3217](https://togithub.com/facebook/zstd/pull/3217) - Fix small file passthrough by [@​cgbur](https://togithub.com/cgbur) in [https://github.com/facebook/zstd/pull/3215](https://togithub.com/facebook/zstd/pull/3215) - Add warning when multi-thread decompression is requested by [@​tomcwang](https://togithub.com/tomcwang) in [https://github.com/facebook/zstd/pull/3208](https://togithub.com/facebook/zstd/pull/3208) - stdin + multiple file fixes by [@​yoniko](https://togithub.com/yoniko) in [https://github.com/facebook/zstd/pull/3222](https://togithub.com/facebook/zstd/pull/3222) - \[AIX] Fixing hash4Ptr for Big Endian Systems by [@​qiongsiwu](https://togithub.com/qiongsiwu) in [https://github.com/facebook/zstd/pull/3227](https://togithub.com/facebook/zstd/pull/3227) - Disallow empty string as argument for --output-dir-flat and --output-dir-mirror by [@​embg](https://togithub.com/embg) in [https://github.com/facebook/zstd/pull/3220](https://togithub.com/facebook/zstd/pull/3220) - Deprecate ZSTD_getDecompressedSize() by [@​terrelln](https://togithub.com/terrelln) in [https://github.com/facebook/zstd/pull/3225](https://togithub.com/facebook/zstd/pull/3225) - \[T124890272] Mark 2 Obsolete Functions(ZSTD_copy\*Ctx) Deprecated in Zstd by [@​mileshu](https://togithub.com/mileshu) in [https://github.com/facebook/zstd/pull/3196](https://togithub.com/facebook/zstd/pull/3196) - fileio_types.h : avoid dependency on mem.h by [@​Cyan4973](https://togithub.com/Cyan4973) in [https://github.com/facebook/zstd/pull/3232](https://togithub.com/facebook/zstd/pull/3232) - fixed: verbose output prints wrong value for `wlog` when doing `--long` by [@​zengyijing](https://togithub.com/zengyijing) in [https://github.com/facebook/zstd/pull/3226](https://togithub.com/facebook/zstd/pull/3226) - Add explicit --pass-through flag and default to enabled for \*cat by [@​terrelln](https://togithub.com/terrelln) in [https://github.com/facebook/zstd/pull/3223](https://togithub.com/facebook/zstd/pull/3223) - Document pass-through behavior by [@​cgbur](https://togithub.com/cgbur) in [https://github.com/facebook/zstd/pull/3242](https://togithub.com/facebook/zstd/pull/3242) - restore combine.sh bash performance while still sticking to POSIX by [@​wahern](https://togithub.com/wahern) in [https://github.com/facebook/zstd/pull/3241](https://togithub.com/facebook/zstd/pull/3241) - Benchmark program for sequence compression API by [@​embg](https://togithub.com/embg) in [https://github.com/facebook/zstd/pull/3257](https://togithub.com/facebook/zstd/pull/3257) - streamline `make clean` list maintenance by adding a `CLEAN` variable by [@​Cyan4973](https://togithub.com/Cyan4973) in [https://github.com/facebook/zstd/pull/3256](https://togithub.com/facebook/zstd/pull/3256) - drop `-E` flag in `sed` by [@​haampie](https://togithub.com/haampie) in [https://github.com/facebook/zstd/pull/3245](https://togithub.com/facebook/zstd/pull/3245) - compress:check more bytes to reduce `ZSTD_count` call by [@​JunHe77](https://togithub.com/JunHe77) in [https://github.com/facebook/zstd/pull/3199](https://togithub.com/facebook/zstd/pull/3199) - build(cmake): improve pkg-config generation by [@​Tachi107](https://togithub.com/Tachi107) in [https://github.com/facebook/zstd/pull/3252](https://togithub.com/facebook/zstd/pull/3252) - Fix for `zstd` CLI accepts bogus values for numeric parameters by [@​ctkhanhly](https://togithub.com/ctkhanhly) in [https://github.com/facebook/zstd/pull/3268](https://togithub.com/facebook/zstd/pull/3268) - ci: test pkg-config file by [@​Tachi107](https://togithub.com/Tachi107) in [https://github.com/facebook/zstd/pull/3267](https://togithub.com/facebook/zstd/pull/3267) - Move ZSTD_DEPRECATED before ZSTDLIB_API/ZSTDLIB_STATIC_API for `clang` by [@​MaskRay](https://togithub.com/MaskRay) in [https://github.com/facebook/zstd/pull/3273](https://togithub.com/facebook/zstd/pull/3273) - Enable OpenSSF Scorecard Action by [@​felixhandte](https://togithub.com/felixhandte) in [https://github.com/facebook/zstd/pull/3277](https://togit

Configuration

📅 Schedule: Branch creation - "every weekend" (UTC), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

This PR has been generated by Mend Renovate. View repository job log here.