Ensuring delta encoding footer blocks are complete And Handle Overflow by @ee-naveen in #387.
Use PLAIN encoding for columns without defined data by @spanglerco in #388.
4.16.0
New
Markdown documentation fully migrated to GitHub Pages. It was becoming slightly unmanageable and also recent GitHub updates made markdown files look awful. Also I kind of wanted to try Writerside by JetBrains, and publish docs with pride ;) @aloneguid
Class deserializer will now skip class properties and not throw an exception if they are missing in the source parquet file. Thanks to @greenlynx in #361.
Column statistics can be read with zero cost without reading the data. Thanks to @mirosuav in #252, #368.
Support for DELTA_BINARY_PACKED encoding on write. This encoding is now default when writing INT32 and INT64 columns. Most of the work done by @ee-naveen in #382.
Improvements
IronCompress was updated to v1.5.1 by @aloneguid.
Fixes
Fix precision issues writing DateTime as milliseconds by @spanglerco in #312.
In DataColumnWriter, RecycableMemoryStream wasn't used in a particular case, and instead MemoryStream was initialized directly. Thanks to @itayfisz in #373.
Bitpacked Hybrid decoder was failing on columns containing exactly one value.
4.15.0
Bugs Fixed
strings must be null by default (#360). Thanks @waf!
New Stuff
You can force optionality of a schema field using the [ParquetRequired] attribute.
ParquetSerializer validates class schema against actual file schema on deserialization and throws a helpful exception, like System.IO.InvalidDataException : property 'Id' is declared as Id (System.String?) but source data has it as Id (System.String) (you can spot the difference in nullability here).
4.14.0
Added support for reading legacy array primitives collection serialized via legacy ParquetConvert class or some other legacy system, thanks to @PablitoCBR. This work was effectively taken from his PR and integrated more natively into this library. Thank you very much!
Fixed deserializing parquet generated by Azure Data Explorer non-native writer by @mcbos in #357.
re-worked build pipeline to separate build and release stage.
use handcrafted release notes file and cut out last version notes with grep/head/tail on release. This is in order to improve release notes experience as autogenerated ones are often of sub-par quality.
Ensuring delta encoding footer blocks are complete And Handle Overflow by @ee-naveen in #387.
Use PLAIN encoding for columns without defined data by @spanglerco in #388.
4.16.0
New
Markdown documentation fully migrated to GitHub Pages. It was becoming slightly unmanageable and also recent GitHub updates made markdown files look awful. Also I kind of wanted to try Writerside by JetBrains, and publish docs with pride ;) @aloneguid
Class deserializer will now skip class properties and not throw an exception if they are missing in the source parquet file. Thanks to @greenlynx in #361.
Column statistics can be read with zero cost without reading the data. Thanks to @mirosuav in #252, #368.
Support for DELTA_BINARY_PACKED encoding on write. This encoding is now default when writing INT32 and INT64 columns. Most of the work done by @ee-naveen in #382.
Improvements
IronCompress was updated to v1.5.1 by @aloneguid.
Fixes
Fix precision issues writing DateTime as milliseconds by @spanglerco in #312.
In DataColumnWriter, RecycableMemoryStream wasn't used in a particular case, and instead MemoryStream was initialized directly. Thanks to @itayfisz in #373.
Bitpacked Hybrid decoder was failing on columns containing exactly one value.
4.15.0
Bugs Fixed
strings must be null by default (#360). Thanks @waf!
New Stuff
You can force optionality of a schema field using the [ParquetRequired] attribute.
ParquetSerializer validates class schema against actual file schema on deserialization and throws a helpful exception, like System.IO.InvalidDataException : property 'Id' is declared as Id (System.String?) but source data has it as Id (System.String) (you can spot the difference in nullability here).
4.14.0
Added support for reading legacy array primitives collection serialized via legacy ParquetConvert class or some other legacy system, thanks to @PablitoCBR. This work was effectively taken from his PR and integrated more natively into this library. Thank you very much!
Fixed deserializing parquet generated by Azure Data Explorer non-native writer by @mcbos in #357.
re-worked build pipeline to separate build and release stage.
use handcrafted release notes file and cut out last version notes with grep/head/tail on release. This is in order to improve release notes experience as autogenerated ones are often of sub-par quality.
4.13.0
Add support for deserializing required strings by @mcbos in #341
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
dependabot[bot]
commented
1 year ago
Bumps Parquet.Net from 4.0.2 to 4.16.1.
Release notes
Sourced from Parquet.Net's releases.
... (truncated)
Changelog
Sourced from Parquet.Net's changelog.
... (truncated)
Commits
61c3119update release notesad5be0fUse PLAIN encoding for columns without defined data63f1c44release notes for 4.16.1058c8faversion bump2dcf362Handle Overflow: Using MSB position to calculate bit width. (#6)23cc2d2Ensuring delta encoding footer blocks are complete.07e7c67version bump6b9c99cupdating release notesfd17b82delta binary packed is now the defaulte806525ULEB128 cleanupMost Recent Ignore Conditions Applied to This Pull Request
| Dependency Name | Ignore Conditions | | --- | --- | | Parquet.Net | [>= 4.1.a, < 4.2] | | Parquet.Net | [>= 4.2.a, < 4.3] | | Parquet.Net | [>= 4.3.a, < 4.4] |Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting
@dependabot rebase.Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show