search

timvw / qv

Quickly view your data
Apache License 2.0
272 stars 16 forks source link

fix(deps): update rust crate deltalake to 0.19 #129

Open renovate[bot] opened 3 months ago

renovate[bot] commented 3 months ago

This PR contains the following updates:

Package Type Update Change
deltalake dependencies minor 0.17 -> 0.19

Release Notes

delta-io/delta.rs (deltalake) ### [`v0.19.0`](https://togithub.com/delta-io/delta.rs/blob/HEAD/CHANGELOG.md#rust-v0190-2024-08-14) [Full Changelog](https://togithub.com/delta-io/delta-rs/compare/rust-v0.18.2...rust-v0.19.0) **Implemented enhancements:** - Only allow squash merge [#​2542](https://togithub.com/delta-io/delta-rs/issues/2542) **Fixed bugs:** - Write also insert change types in writer CDC [#​2750](https://togithub.com/delta-io/delta-rs/issues/2750) - Regression in Python multiprocessing support [#​2744](https://togithub.com/delta-io/delta-rs/issues/2744) - SchemaError occurs during table optimisation after upgrade to v0.18.1 [#​2731](https://togithub.com/delta-io/delta-rs/issues/2731) - AWS WebIdentityToken exposure in log files [#​2719](https://togithub.com/delta-io/delta-rs/issues/2719) - Write performance degrades with multiple writers [#​2683](https://togithub.com/delta-io/delta-rs/issues/2683) - Write monotonic sequence, but read is non monotonic [#​2659](https://togithub.com/delta-io/delta-rs/issues/2659) - Python `write_deltalake` with `schema_mode="merge"` casts types [#​2642](https://togithub.com/delta-io/delta-rs/issues/2642) - Newest docs (potentially) not released [#​2587](https://togithub.com/delta-io/delta-rs/issues/2587) - CDC is not generated for Structs and Lists [#​2568](https://togithub.com/delta-io/delta-rs/issues/2568) **Closed issues:** - delete_dir bug [#​2713](https://togithub.com/delta-io/delta-rs/issues/2713) **Merged pull requests:** - chore: fix a bunch of clippy lints and re-enable tests [#​2773](https://togithub.com/delta-io/delta-rs/pull/2773) ([rtyler](https://togithub.com/rtyler)) - feat: more economic data skipping with datafusion [#​2772](https://togithub.com/delta-io/delta-rs/pull/2772) ([roeap](https://togithub.com/roeap)) - chore: prepare the next notable release of 0.19.0 [#​2768](https://togithub.com/delta-io/delta-rs/pull/2768) ([rtyler](https://togithub.com/rtyler)) - feat: restore the TryFrom for DeltaTablePartition [#​2767](https://togithub.com/delta-io/delta-rs/pull/2767) ([rtyler](https://togithub.com/rtyler)) - feat: fail fast on forked process [#​2765](https://togithub.com/delta-io/delta-rs/pull/2765) ([Tom-Newton](https://togithub.com/Tom-Newton)) - perf: early stop if all values in arr are null [#​2764](https://togithub.com/delta-io/delta-rs/pull/2764) ([ion-elgreco](https://togithub.com/ion-elgreco)) - fix(python, rust): don't flatten fields during cdf read [#​2763](https://togithub.com/delta-io/delta-rs/pull/2763) ([ion-elgreco](https://togithub.com/ion-elgreco)) - chore: upgrade to datafusion 41 [#​2761](https://togithub.com/delta-io/delta-rs/pull/2761) ([rtyler](https://togithub.com/rtyler)) - fix(python, rust): cdc in writer not creating inserts [#​2751](https://togithub.com/delta-io/delta-rs/pull/2751) ([ion-elgreco](https://togithub.com/ion-elgreco)) - feat: improved test fixtures [#​2749](https://togithub.com/delta-io/delta-rs/pull/2749) ([roeap](https://togithub.com/roeap)) - feat: introduce CDC generation for merge operations [#​2747](https://togithub.com/delta-io/delta-rs/pull/2747) ([rtyler](https://togithub.com/rtyler)) - docs: fix broken link in docs [#​2746](https://togithub.com/delta-io/delta-rs/pull/2746) ([astrojuanlu](https://togithub.com/astrojuanlu)) - chore: update delta_kernel to 0.3.0 [#​2742](https://togithub.com/delta-io/delta-rs/pull/2742) ([alexwilcoxson-rel](https://togithub.com/alexwilcoxson-rel)) - chore: add to code_owner crates [#​2741](https://togithub.com/delta-io/delta-rs/pull/2741) ([ion-elgreco](https://togithub.com/ion-elgreco)) - chore: update changelog and versions for next release [#​2740](https://togithub.com/delta-io/delta-rs/pull/2740) ([rtyler](https://togithub.com/rtyler)) - feat(python, rust): arrow large/view types passthrough, rust default engine [#​2738](https://togithub.com/delta-io/delta-rs/pull/2738) ([ion-elgreco](https://togithub.com/ion-elgreco)) - fix: column parsing to include nested columns and enclosing char [#​2737](https://togithub.com/delta-io/delta-rs/pull/2737) ([gtrawinski](https://togithub.com/gtrawinski)) ### [`v0.18.2`](https://togithub.com/delta-io/delta.rs/blob/HEAD/CHANGELOG.md#rust-v0182-2024-08-07) [Full Changelog](https://togithub.com/delta-io/delta-rs/compare/rust-v0.18.1...rust-v0.18.2) **Implemented enhancements:** - Choose which columns to store min/max values for [#​2709](https://togithub.com/delta-io/delta-rs/issues/2709) - Projection pushdown for load_cdf [#​2681](https://togithub.com/delta-io/delta-rs/issues/2681) - Way to check if Delta table exists at specified path [#​2662](https://togithub.com/delta-io/delta-rs/issues/2662) - Support HDFS via hdfs-native package [#​2611](https://togithub.com/delta-io/delta-rs/issues/2611) - Deletion `_change_type` does not appear in change data feed [#​2579](https://togithub.com/delta-io/delta-rs/issues/2579) - Could you please explain in the README what "Deltalake" is for the uninitiated? [#​2523](https://togithub.com/delta-io/delta-rs/issues/2523) - Discuss: Allow protocol change during write actions [#​2444](https://togithub.com/delta-io/delta-rs/issues/2444) - Support for Arrow PyCapsule interface [#​2376](https://togithub.com/delta-io/delta-rs/issues/2376) **Fixed bugs:** - Slow add_actions.to_pydict for tables with large number of columns, impacting read performance [#​2733](https://togithub.com/delta-io/delta-rs/issues/2733) - append is deleting records [#​2716](https://togithub.com/delta-io/delta-rs/issues/2716) - segmentation fault - Python 3.10 on Mac M3 [#​2706](https://togithub.com/delta-io/delta-rs/issues/2706) - Failure to delete dir and files [#​2703](https://togithub.com/delta-io/delta-rs/issues/2703) - DeltaTable.from_data_catalog not working [#​2699](https://togithub.com/delta-io/delta-rs/issues/2699) - Project should use the same version of `ruff` in the `lint` stage of `python_build.yml` as in `pyproject.toml` [#​2678](https://togithub.com/delta-io/delta-rs/issues/2678) - un-tracked columns are giving json error when pyarrow schema have feild with nullable=False and create_checkpoint is trigged [#​2675](https://togithub.com/delta-io/delta-rs/issues/2675) - \[BUG]write_delta({'custom_metadata':str}) cannot be converted. str to pyDict error (0.18.2\_DeltaPython/Windows10) [#​2697](https://togithub.com/delta-io/delta-rs/issues/2697) - Pyarrow engine not supporting schema overwrite with Append mode [#​2654](https://togithub.com/delta-io/delta-rs/issues/2654) - `deltalake-core` version re-exported by `deltalake` different than versions used by `deltalake-azure` and `deltalake-gcp` [#​2647](https://togithub.com/delta-io/delta-rs/issues/2647) - i32 limit in JSON stats [#​2646](https://togithub.com/delta-io/delta-rs/issues/2646) - Rust writer not encoding correct URL for partitions in delta table [#​2634](https://togithub.com/delta-io/delta-rs/issues/2634) - Large Types breaks merge predicate pruning [#​2632](https://togithub.com/delta-io/delta-rs/issues/2632) - Getting error when converting a partitioned parquet table to delta table [#​2626](https://togithub.com/delta-io/delta-rs/issues/2626) - Arrow: Parquet does not support writing empty structs when creating checkpoint [#​2622](https://togithub.com/delta-io/delta-rs/issues/2622) - InvalidTableLocation("Unknown scheme: gs") on 0.18.0 [#​2610](https://togithub.com/delta-io/delta-rs/issues/2610) - Unable to read delta table created using Uniform [#​2578](https://togithub.com/delta-io/delta-rs/issues/2578) - schema merging doesn't work when overwriting with a predicate [#​2567](https://togithub.com/delta-io/delta-rs/issues/2567) - Not working in AWS Lambda (0.16.2 - 0.17.4) OSError: Generic S3 error [#​2511](https://togithub.com/delta-io/delta-rs/issues/2511) - DataFusion filter on partition column doesn't work. (when the phsical schema ordering is different to logical one) [#​2494](https://togithub.com/delta-io/delta-rs/issues/2494) - Creating checkpoints for tables with missing column stats results in Err [#​2493](https://togithub.com/delta-io/delta-rs/issues/2493) - Cannot merge to a table with a timestamp column after upgrading delta-rs [#​2478](https://togithub.com/delta-io/delta-rs/issues/2478) - Azure AD Auth fails on ARM64 [#​2475](https://togithub.com/delta-io/delta-rs/issues/2475) - Generic S3 error: Error after 0 retries ... Broken pipe (os error 32) [#​2403](https://togithub.com/delta-io/delta-rs/issues/2403) - write_deltalake identifies large_string as datatype even though string is set in schema [#​2374](https://togithub.com/delta-io/delta-rs/issues/2374) - Inconsistent arrow timestamp type breaks datafusion query [#​2341](https://togithub.com/delta-io/delta-rs/issues/2341) **Closed issues:** - Unable to write new partitions with type timestamp on tables created with delta-rs 0.10.0 [#​2631](https://togithub.com/delta-io/delta-rs/issues/2631) **Merged pull requests:** - fix: schema adapter doesn't map partial batches correctly [#​2735](https://togithub.com/delta-io/delta-rs/pull/2735) ([alexwilcoxson-rel](https://togithub.com/alexwilcoxson-rel)) - perf: grab file size in rust [#​2734](https://togithub.com/delta-io/delta-rs/pull/2734) ([ion-elgreco](https://togithub.com/ion-elgreco)) - feat: use logical plan in update, refactor/simplify CDCTracker [#​2727](https://togithub.com/delta-io/delta-rs/pull/2727) ([ion-elgreco](https://togithub.com/ion-elgreco)) - feat: use logical plan in delete, delta planner refactoring [#​2725](https://togithub.com/delta-io/delta-rs/pull/2725) ([ion-elgreco](https://togithub.com/ion-elgreco)) - chore: try an alternative docke compose invocation syntax [#​2724](https://togithub.com/delta-io/delta-rs/pull/2724) ([rtyler](https://togithub.com/rtyler)) - fix(python, rust): use input schema to get correct schema in cdf reads [#​2723](https://togithub.com/delta-io/delta-rs/pull/2723) ([ion-elgreco](https://togithub.com/ion-elgreco)) - feat(python, rust): cdc write-support for `overwrite` and `replacewhere` writes [#​2722](https://togithub.com/delta-io/delta-rs/pull/2722) ([ion-elgreco](https://togithub.com/ion-elgreco)) - feat(python, rust): cdc write-support for `delete` operation [#​2721](https://togithub.com/delta-io/delta-rs/pull/2721) ([ion-elgreco](https://togithub.com/ion-elgreco)) - chore: enabling actions for merge groups [#​2718](https://togithub.com/delta-io/delta-rs/pull/2718) ([rtyler](https://togithub.com/rtyler)) - perf: apply projection when reading checkpoint parquet [#​2717](https://togithub.com/delta-io/delta-rs/pull/2717) ([alexwilcoxson-rel](https://togithub.com/alexwilcoxson-rel)) - feat(python): add DeltaTable.is_deltatable static method ([#​2662](https://togithub.com/delta-io/delta.rs/issues/2662)) [#​2715](https://togithub.com/delta-io/delta-rs/pull/2715) ([omkar-foss](https://togithub.com/omkar-foss)) - chore: prepare python release 0.18.3 [#​2707](https://togithub.com/delta-io/delta-rs/pull/2707) ([ion-elgreco](https://togithub.com/ion-elgreco)) - fix(python, rust): use url encoder when encoding partition values [#​2705](https://togithub.com/delta-io/delta-rs/pull/2705) ([ion-elgreco](https://togithub.com/ion-elgreco)) - feat(python, rust): add projection in CDF reads [#​2704](https://togithub.com/delta-io/delta-rs/pull/2704) ([ion-elgreco](https://togithub.com/ion-elgreco)) - fix: ensure DataFusion SessionState Parquet options are applied to DeltaScan [#​2702](https://togithub.com/delta-io/delta-rs/pull/2702) ([alexwilcoxson-rel](https://togithub.com/alexwilcoxson-rel)) - chore: refactor `write_deltalake` in `writer.py` [#​2695](https://togithub.com/delta-io/delta-rs/pull/2695) ([fpgmaas](https://togithub.com/fpgmaas)) - fix(python): empty dataset fix for "pyarrow" engine [#​2689](https://togithub.com/delta-io/delta-rs/pull/2689) ([sherlockbeard](https://togithub.com/sherlockbeard)) - chore: add test coverage command to `Makefile` [#​2688](https://togithub.com/delta-io/delta-rs/pull/2688) ([fpgmaas](https://togithub.com/fpgmaas)) - chore: create separate action to setup python and rust in the cicd pipeline [#​2687](https://togithub.com/delta-io/delta-rs/pull/2687) ([fpgmaas](https://togithub.com/fpgmaas)) - fix: update delta kernel version [#​2685](https://togithub.com/delta-io/delta-rs/pull/2685) ([jeppe742](https://togithub.com/jeppe742)) - chore: update README.md [#​2684](https://togithub.com/delta-io/delta-rs/pull/2684) ([veronewra](https://togithub.com/veronewra)) - fix(rust,python): checkpoint with column nullable false [#​2680](https://togithub.com/delta-io/delta-rs/pull/2680) ([sherlockbeard](https://togithub.com/sherlockbeard)) - chore: pin `ruff` and `mypy` versions in the `lint` stage in the CI pipeline [#​2679](https://togithub.com/delta-io/delta-rs/pull/2679) ([fpgmaas](https://togithub.com/fpgmaas)) - chore: enable `RUF` ruleset for `ruff` [#​2677](https://togithub.com/delta-io/delta-rs/pull/2677) ([fpgmaas](https://togithub.com/fpgmaas)) - chore: remove stale code for conditional import of `Literal` [#​2676](https://togithub.com/delta-io/delta-rs/pull/2676) ([fpgmaas](https://togithub.com/fpgmaas)) - chore: remove references to black from the project [#​2674](https://togithub.com/delta-io/delta-rs/pull/2674) ([fpgmaas](https://togithub.com/fpgmaas)) - chore: bump ruff to 0.5.2 [#​2673](https://togithub.com/delta-io/delta-rs/pull/2673) ([fpgmaas](https://togithub.com/fpgmaas)) - chore: improve contributing.md [#​2672](https://togithub.com/delta-io/delta-rs/pull/2672) ([fpgmaas](https://togithub.com/fpgmaas)) - feat: support userMetadata in CommitInfo [#​2670](https://togithub.com/delta-io/delta-rs/pull/2670) ([jkylling](https://togithub.com/jkylling)) - chore: upgrade to datafusion 40 [#​2661](https://togithub.com/delta-io/delta-rs/pull/2661) ([rtyler](https://togithub.com/rtyler)) - docs: improve navigation fixes [#​2660](https://togithub.com/delta-io/delta-rs/pull/2660) ([avriiil](https://togithub.com/avriiil)) - docs: add integration docs for s3 backend [#​2658](https://togithub.com/delta-io/delta-rs/pull/2658) ([avriiil](https://togithub.com/avriiil)) - docs: fix bullets on hdfs docs [#​2653](https://togithub.com/delta-io/delta-rs/pull/2653) ([Kimahriman](https://togithub.com/Kimahriman)) - ci: update CODEOWNERS [#​2650](https://togithub.com/delta-io/delta-rs/pull/2650) ([hntd187](https://togithub.com/hntd187)) - feat(rust): fix size_in_bytes in last_checkpoint\_ to i64 [#​2649](https://togithub.com/delta-io/delta-rs/pull/2649) ([sherlockbeard](https://togithub.com/sherlockbeard)) - chore: increase subcrate versions [#​2648](https://togithub.com/delta-io/delta-rs/pull/2648) ([rtyler](https://togithub.com/rtyler)) - chore: missed one macos runner reference in actions [#​2645](https://togithub.com/delta-io/delta-rs/pull/2645) ([rtyler](https://togithub.com/rtyler)) - chore: add a reproduction case for merge failures with struct\ [#​2644](https://togithub.com/delta-io/delta-rs/pull/2644) ([rtyler](https://togithub.com/rtyler)) - chore: remove macos builders from pull request flow [#​2638](https://togithub.com/delta-io/delta-rs/pull/2638) ([rtyler](https://togithub.com/rtyler)) - fix: enable parquet pushdown for DeltaScan via TableProvider impl for DeltaTable (rebase) [#​2637](https://togithub.com/delta-io/delta-rs/pull/2637) ([rtyler](https://togithub.com/rtyler)) - chore: fix documentation generation with a pin of griffe [#​2636](https://togithub.com/delta-io/delta-rs/pull/2636) ([rtyler](https://togithub.com/rtyler)) - fix(python): fixed large_dtype to schema convert [#​2635](https://togithub.com/delta-io/delta-rs/pull/2635) ([sherlockbeard](https://togithub.com/sherlockbeard)) - fix(rust, python): fix writing empty structs when creating checkpoint [#​2627](https://togithub.com/delta-io/delta-rs/pull/2627) ([sherlockbeard](https://togithub.com/sherlockbeard)) - fix(rust, python): fix merge schema with overwrite [#​2623](https://togithub.com/delta-io/delta-rs/pull/2623) ([sherlockbeard](https://togithub.com/sherlockbeard)) - chore: bump python 0.18.2 [#​2621](https://togithub.com/delta-io/delta-rs/pull/2621) ([ion-elgreco](https://togithub.com/ion-elgreco)) - feat: report DataFusion metrics for DeltaScan [#​2617](https://togithub.com/delta-io/delta-rs/pull/2617) ([alexwilcoxson-rel](https://togithub.com/alexwilcoxson-rel)) - feat(rust,python): cast each parquet file to delta schema [#​2615](https://togithub.com/delta-io/delta-rs/pull/2615) ([HawaiianSpork](https://togithub.com/HawaiianSpork)) - fix(rust): inconsistent order of partitioning columns ([#​2494](https://togithub.com/delta-io/delta.rs/issues/2494)) [#​2614](https://togithub.com/delta-io/delta-rs/pull/2614) ([aditanase](https://togithub.com/aditanase)) - docs: add Daft writer [#​2594](https://togithub.com/delta-io/delta-rs/pull/2594) ([avriiil](https://togithub.com/avriiil)) - feat(python, rust): `add column` operation [#​2562](https://togithub.com/delta-io/delta-rs/pull/2562) ([ion-elgreco](https://togithub.com/ion-elgreco)) - fix: change arrow map root name to follow with parquet root name [#​2538](https://togithub.com/delta-io/delta-rs/pull/2538) ([sclmn](https://togithub.com/sclmn)) - feat(python): handle PyCapsule interface objects in write_deltalake [#​2534](https://togithub.com/delta-io/delta-rs/pull/2534) ([kylebarron](https://togithub.com/kylebarron)) - feat: improve merge performance by using predicate non-partition columns min/max for prefiltering [#​2513](https://togithub.com/delta-io/delta-rs/pull/2513) ([JonasDev1](https://togithub.com/JonasDev1)) - feat(python, rust): cleanup expired logs post-commit hook [#​2459](https://togithub.com/delta-io/delta-rs/pull/2459) ([ion-elgreco](https://togithub.com/ion-elgreco)) ### [`v0.18.0`](https://togithub.com/delta-io/delta.rs/blob/HEAD/CHANGELOG.md#rust-v0180-2024-06-12) [Full Changelog](https://togithub.com/delta-io/delta-rs/compare/rust-v0.17.3...rust-v0.18.0) **Implemented enhancements:** - documentation: concurrent writes for non-S3 backends [#​2556](https://togithub.com/delta-io/delta-rs/issues/2556) - pyarrow options for `write_delta` [#​2515](https://togithub.com/delta-io/delta-rs/issues/2515) - \[deltalake_aws] Allow configuring separate endpoints for S3 and DynamoDB clients. [#​2498](https://togithub.com/delta-io/delta-rs/issues/2498) - Include file stats when converting a parquet directory to a Delta table [#​2490](https://togithub.com/delta-io/delta-rs/issues/2490) - Adopt the delta kernel types [#​2489](https://togithub.com/delta-io/delta-rs/issues/2489) **Fixed bugs:** - `raise_if_not_exists` for properties not configurable on CreateBuilder [#​2564](https://togithub.com/delta-io/delta-rs/issues/2564) - write_deltalake with rust engine fails when mode is append and overwrite schema is enabled [#​2553](https://togithub.com/delta-io/delta-rs/issues/2553) - Running the basic_operations examples fails with ` Error: Transaction { source: WriterFeaturesRequired(TimestampWithoutTimezone) `} [#​2552](https://togithub.com/delta-io/delta-rs/issues/2552) - invalid peer certificate: BadSignature when connecting to s3 from arm64/aarch64 [#​2551](https://togithub.com/delta-io/delta-rs/issues/2551) - load_cdf() issue : Generic S3 error: request or response body error: operation timed out [#​2549](https://togithub.com/delta-io/delta-rs/issues/2549) - write_deltalake fails on Databricks volume [#​2540](https://togithub.com/delta-io/delta-rs/issues/2540) - Getting "Microsoft Azure Error: Operation timed out" when trying to retrieve big files [#​2537](https://togithub.com/delta-io/delta-rs/issues/2537) - Impossible to append to a DeltaTable with float data type on RHEL [#​2520](https://togithub.com/delta-io/delta-rs/issues/2520) - Creating DeltaTable object slow [#​2518](https://togithub.com/delta-io/delta-rs/issues/2518) - `write_deltalake` throws parser error when using `rust` engine and big decimals [#​2510](https://togithub.com/delta-io/delta-rs/issues/2510) - TypeError: Object of type int64 is not JSON serializable when writing using a Pandas dataframe [#​2501](https://togithub.com/delta-io/delta-rs/issues/2501) - unable to read delta table when table contains both null and non-null add stats [#​2477](https://togithub.com/delta-io/delta-rs/issues/2477) - Commits on WriteMode::MergeSchema cause table metadata corruption [#​2468](https://togithub.com/delta-io/delta-rs/issues/2468) - S3 object store always returns IMDS warnings [#​2460](https://togithub.com/delta-io/delta-rs/issues/2460) - File skipping according to documentation [#​2427](https://togithub.com/delta-io/delta-rs/issues/2427) - LockClientError [#​2379](https://togithub.com/delta-io/delta-rs/issues/2379) - get_app_transaction_version() returns wrong result [#​2340](https://togithub.com/delta-io/delta-rs/issues/2340) - Property setting in `create` is not handled correctly [#​2247](https://togithub.com/delta-io/delta-rs/issues/2247) - Handling of decimals in scientific notation [#​2221](https://togithub.com/delta-io/delta-rs/issues/2221) - Unable to append to delta table without datafusion feature [#​2204](https://togithub.com/delta-io/delta-rs/issues/2204) - Decimal Column with Value 0 Causes Failure in Python Binding [#​2193](https://togithub.com/delta-io/delta-rs/issues/2193) **Merged pull requests:** - docs: improve S3 access docs [#​2589](https://togithub.com/delta-io/delta-rs/pull/2589) ([avriiil](https://togithub.com/avriiil)) - chore: bump macOS runners, maybe resolve import error [#​2588](https://togithub.com/delta-io/delta-rs/pull/2588) ([ion-elgreco](https://togithub.com/ion-elgreco)) - chore: bump to datafusion 39, arrow 52, pyo3 0.21 [#​2581](https://togithub.com/delta-io/delta-rs/pull/2581) ([abhiaagarwal](https://togithub.com/abhiaagarwal)) - feat: add custom dynamodb endpoint configuration [#​2575](https://togithub.com/delta-io/delta-rs/pull/2575) ([hnaoto](https://togithub.com/hnaoto)) - fix: consistently use raise_if_key_not_exists in CreateBuilder [#​2569](https://togithub.com/delta-io/delta-rs/pull/2569) ([vegarsti](https://togithub.com/vegarsti)) - fix: add raise_if_key_not_exists to CreateBuilder [#​2565](https://togithub.com/delta-io/delta-rs/pull/2565) ([vegarsti](https://togithub.com/vegarsti)) - docs: dt.delete add context + api docs link [#​2560](https://togithub.com/delta-io/delta-rs/pull/2560) ([avriiil](https://togithub.com/avriiil)) - fix: update deltalake crate examples for crate layout and TimestampNtz [#​2559](https://togithub.com/delta-io/delta-rs/pull/2559) ([jhoekx](https://togithub.com/jhoekx)) - docs: clarify locking mechanism requirement for S3 [#​2558](https://togithub.com/delta-io/delta-rs/pull/2558) ([inigohidalgo](https://togithub.com/inigohidalgo)) - fix: remove deprecated overwrite_schema configuration which has incorrect behavior [#​2554](https://togithub.com/delta-io/delta-rs/pull/2554) ([rtyler](https://togithub.com/rtyler)) - fix: clippy warnings [#​2548](https://togithub.com/delta-io/delta-rs/pull/2548) ([imor](https://togithub.com/imor)) - docs: dask write syntax fix [#​2543](https://togithub.com/delta-io/delta-rs/pull/2543) ([avriiil](https://togithub.com/avriiil)) - fix: cast support fields nested in lists and maps [#​2541](https://togithub.com/delta-io/delta-rs/pull/2541) ([HawaiianSpork](https://togithub.com/HawaiianSpork)) - feat: implement transaction identifiers - continued [#​2539](https://togithub.com/delta-io/delta-rs/pull/2539) ([roeap](https://togithub.com/roeap)) - docs: pull delta from conda not pip [#​2535](https://togithub.com/delta-io/delta-rs/pull/2535) ([avriiil](https://togithub.com/avriiil)) - chore: expose `files_by_partition` to public api [#​2533](https://togithub.com/delta-io/delta-rs/pull/2533) ([edmondop](https://togithub.com/edmondop)) - chore: bump python 0.17.5 [#​2531](https://togithub.com/delta-io/delta-rs/pull/2531) ([ion-elgreco](https://togithub.com/ion-elgreco)) - feat(rust): make PartitionWriter public [#​2525](https://togithub.com/delta-io/delta-rs/pull/2525) ([adriangb](https://togithub.com/adriangb)) - fix: msrv in workspace [#​2524](https://togithub.com/delta-io/delta-rs/pull/2524) ([roeap](https://togithub.com/roeap)) - chore: fixing some clips [#​2521](https://togithub.com/delta-io/delta-rs/pull/2521) ([rtyler](https://togithub.com/rtyler)) - fix: enable field_with_name to support nested fields with '.' delimiter [#​2519](https://togithub.com/delta-io/delta-rs/pull/2519) ([alexwilcoxson-rel](https://togithub.com/alexwilcoxson-rel)) - chore: tidying up builds without datafusion feature and clippy [#​2516](https://togithub.com/delta-io/delta-rs/pull/2516) ([rtyler](https://togithub.com/rtyler)) - fix(python): release GIL on most operations [#​2512](https://togithub.com/delta-io/delta-rs/pull/2512) ([adriangb](https://togithub.com/adriangb)) - docs: fix typo [#​2508](https://togithub.com/delta-io/delta-rs/pull/2508) ([avriiil](https://togithub.com/avriiil)) - fix(rust, python): fixed differences in storage options between log and object stores [#​2500](https://togithub.com/delta-io/delta-rs/pull/2500) ([mightyshazam](https://togithub.com/mightyshazam)) - docs: improve daft integration docs [#​2496](https://togithub.com/delta-io/delta-rs/pull/2496) ([avriiil](https://togithub.com/avriiil)) - feat: adopt kernel schema types [#​2495](https://togithub.com/delta-io/delta-rs/pull/2495) ([roeap](https://togithub.com/roeap)) - feat: add stats to convert-to-delta operation [#​2491](https://togithub.com/delta-io/delta-rs/pull/2491) ([gruuya](https://togithub.com/gruuya)) - fix(python, rust): region lookup wasn't working correctly for dynamo [#​2488](https://togithub.com/delta-io/delta-rs/pull/2488) ([mightyshazam](https://togithub.com/mightyshazam)) - feat: introduce CDC write-side support for the Update operations [#​2486](https://togithub.com/delta-io/delta-rs/pull/2486) ([rtyler](https://togithub.com/rtyler)) - fix(python): reuse state in `to_pyarrow_dataset` [#​2485](https://togithub.com/delta-io/delta-rs/pull/2485) ([ion-elgreco](https://togithub.com/ion-elgreco)) - fix: check to see if the file exists before attempting to rename [#​2482](https://togithub.com/delta-io/delta-rs/pull/2482) ([rtyler](https://togithub.com/rtyler)) - fix(python, rust): use new schema for stats parsing instead of old [#​2480](https://togithub.com/delta-io/delta-rs/pull/2480) ([ion-elgreco](https://togithub.com/ion-elgreco)) - fix(rust): unable to read delta table when table contains both null and non-null add stats [#​2476](https://togithub.com/delta-io/delta-rs/pull/2476) ([yjshen](https://togithub.com/yjshen)) - chore: update the changelog to include rust-v0.17.3 [#​2473](https://togithub.com/delta-io/delta-rs/pull/2473) ([rtyler](https://togithub.com/rtyler)) - chore: a bunch of tweaks to get releases out the door [#​2472](https://togithub.com/delta-io/delta-rs/pull/2472) ([rtyler](https://togithub.com/rtyler)) - chore: bump the core crate for its next release [#​2470](https://togithub.com/delta-io/delta-rs/pull/2470) ([rtyler](https://togithub.com/rtyler)) - fix: return unsupported error for merging schemas in the presence of partition columns [#​2469](https://togithub.com/delta-io/delta-rs/pull/2469) ([emcake](https://togithub.com/emcake)) - feat(python): add parameter to DeltaTable.to_pyarrow_dataset() [#​2465](https://togithub.com/delta-io/delta-rs/pull/2465) ([adriangb](https://togithub.com/adriangb)) - feat(python, rust): add OBJECT_STORE_CONCURRENCY_LIMIT setting for ObjectStoreFactory [#​2458](https://togithub.com/delta-io/delta-rs/pull/2458) ([vigimite](https://togithub.com/vigimite)) - fix(rust): handle 429 from GCS [#​2454](https://togithub.com/delta-io/delta-rs/pull/2454) ([adriangb](https://togithub.com/adriangb)) - fix(python): reuse table state in write engine [#​2453](https://togithub.com/delta-io/delta-rs/pull/2453) ([ion-elgreco](https://togithub.com/ion-elgreco)) - fix(rust): implement abort commit for S3DynamoDBLogStore [#​2452](https://togithub.com/delta-io/delta-rs/pull/2452) ([PeterKeDer](https://togithub.com/PeterKeDer)) - fix(python, rust): check timestamp_ntz in nested fields, add check_can_write in pyarrow writer [#​2443](https://togithub.com/delta-io/delta-rs/pull/2443) ([ion-elgreco](https://togithub.com/ion-elgreco)) - fix(python, rust): remove imds calls from profile auth and region [#​2442](https://togithub.com/delta-io/delta-rs/pull/2442) ([mightyshazam](https://togithub.com/mightyshazam)) - fix(python, rust): use from_name during column projection creation [#​2441](https://togithub.com/delta-io/delta-rs/pull/2441) ([ion-elgreco](https://togithub.com/ion-elgreco)) - chore: bump python for 0.17 release [#​2439](https://togithub.com/delta-io/delta-rs/pull/2439) ([ion-elgreco](https://togithub.com/ion-elgreco)) - fix(python,rust): missing remove actions during `create_or_replace` [#​2437](https://togithub.com/delta-io/delta-rs/pull/2437) ([ion-elgreco](https://togithub.com/ion-elgreco)) - chore: introduce the Operation trait to enforce consistency between operations [#​2435](https://togithub.com/delta-io/delta-rs/pull/2435) ([rtyler](https://togithub.com/rtyler)) - fix(python): load_as_version with datetime object with no timezone specified [#​2429](https://togithub.com/delta-io/delta-rs/pull/2429) ([t1g0rz](https://togithub.com/t1g0rz)) - feat(python, rust): respect column stats collection configurations [#​2428](https://togithub.com/delta-io/delta-rs/pull/2428) ([ion-elgreco](https://togithub.com/ion-elgreco)) - feat: lazy static runtime in python [#​2424](https://togithub.com/delta-io/delta-rs/pull/2424) ([ion-elgreco](https://togithub.com/ion-elgreco)) - feat: implement repartitioned for DeltaScan [#​2421](https://togithub.com/delta-io/delta-rs/pull/2421) ([jkylling](https://togithub.com/jkylling)) - fix: return error when checkpoints and metadata get out of sync [#​2406](https://togithub.com/delta-io/delta-rs/pull/2406) ([esarili](https://togithub.com/esarili)) - fix(rust): stats_parsed has different number of records with stats [#​2405](https://togithub.com/delta-io/delta-rs/pull/2405) ([yjshen](https://togithub.com/yjshen)) - docs: add Daft integration [#​2402](https://togithub.com/delta-io/delta-rs/pull/2402) ([avriiil](https://togithub.com/avriiil)) - feat(rust): advance state in post commit [#​2396](https://togithub.com/delta-io/delta-rs/pull/2396) ([ion-elgreco](https://togithub.com/ion-elgreco)) - chore(rust): bump arrow v51 and datafusion v37.1 [#​2395](https://togithub.com/delta-io/delta-rs/pull/2395) ([lasantosr](https://togithub.com/lasantosr)) - docs: document required aws permissions [#​2393](https://togithub.com/delta-io/delta-rs/pull/2393) ([ale-rinaldi](https://togithub.com/ale-rinaldi)) - feat(rust): post commit hook (v2), create checkpoint hook [#​2391](https://togithub.com/delta-io/delta-rs/pull/2391) ([ion-elgreco](https://togithub.com/ion-elgreco)) - fix: time travel when checkpointed and logs removed [#​2389](https://togithub.com/delta-io/delta-rs/pull/2389) ([ion-elgreco](https://togithub.com/ion-elgreco)) - fix(rust): remove flush after writing every batch [#​2387](https://togithub.com/delta-io/delta-rs/pull/2387) ([PeterKeDer](https://togithub.com/PeterKeDer)) - feat: added configuration variables to handle EC2 metadata service [#​2385](https://togithub.com/delta-io/delta-rs/pull/2385) ([mightyshazam](https://togithub.com/mightyshazam)) - fix(rust): timestamp deserialization format, missing type [#​2383](https://togithub.com/delta-io/delta-rs/pull/2383) ([ion-elgreco](https://togithub.com/ion-elgreco)) - chore: bump chrono [#​2372](https://togithub.com/delta-io/delta-rs/pull/2372) ([universalmind303](https://togithub.com/universalmind303)) - chore: bump python 0.16.4 [#​2371](https://togithub.com/delta-io/delta-rs/pull/2371) ([ion-elgreco](https://togithub.com/ion-elgreco)) - fix: add snappy compression on checkpoint files [#​2365](https://togithub.com/delta-io/delta-rs/pull/2365) ([ion-elgreco](https://togithub.com/ion-elgreco)) - fix: add config for parquet pushdown on delta scan [#​2364](https://togithub.com/delta-io/delta-rs/pull/2364) ([Blajda](https://togithub.com/Blajda)) - fix(python,rust): optimize compact on schema evolved table [#​2358](https://togithub.com/delta-io/delta-rs/pull/2358) ([ion-elgreco](https://togithub.com/ion-elgreco)) - fix(python, rust): expr parsing date/timestamp [#​2357](https://togithub.com/delta-io/delta-rs/pull/2357) ([ion-elgreco](https://togithub.com/ion-elgreco)) - fix: remove tmp files in cleanup_metadata [#​2356](https://togithub.com/delta-io/delta-rs/pull/2356) ([ion-elgreco](https://togithub.com/ion-elgreco)) - fix: make struct fields nullable in stats schema [#​2346](https://togithub.com/delta-io/delta-rs/pull/2346) ([qinix](https://togithub.com/qinix)) - fix(rust): adhere to protocol for Decimal [#​2332](https://togithub.com/delta-io/delta-rs/pull/2332) ([ion-elgreco](https://togithub.com/ion-elgreco)) - fix(rust): raise schema mismatch when decimal is not subset [#​2330](https://togithub.com/delta-io/delta-rs/pull/2330) ([ion-elgreco](https://togithub.com/ion-elgreco)) - feat(rust): derive Copy on some public enums [#​2329](https://togithub.com/delta-io/delta-rs/pull/2329) ([lasantosr](https://togithub.com/lasantosr)) - fix: merge pushdown handling [#​2326](https://togithub.com/delta-io/delta-rs/pull/2326) ([Blajda](https://togithub.com/Blajda)) - fix: merge concurrency control [#​2324](https://togithub.com/delta-io/delta-rs/pull/2324) ([ion-elgreco](https://togithub.com/ion-elgreco)) - Revert 2291 merge predicate fix [#​2323](https://togithub.com/delta-io/delta-rs/pull/2323) ([Blajda](https://togithub.com/Blajda)) - fix: try to fix timeouts [#​2318](https://togithub.com/delta-io/delta-rs/pull/2318) ([ion-elgreco](https://togithub.com/ion-elgreco)) - fix(rust): serialize MetricDetails from compaction runs to a string [#​2317](https://togithub.com/delta-io/delta-rs/pull/2317) ([liamphmurphy](https://togithub.com/liamphmurphy)) - docs: add example in to_pyarrow_dataset [#​2315](https://togithub.com/delta-io/delta-rs/pull/2315) ([ion-elgreco](https://togithub.com/ion-elgreco)) - fix(python): wrong batch size [#​2314](https://togithub.com/delta-io/delta-rs/pull/2314) ([ion-elgreco](https://togithub.com/ion-elgreco)) - chore: object store 0.9.1 [#​2311](https://togithub.com/delta-io/delta-rs/pull/2311) ([ion-elgreco](https://togithub.com/ion-elgreco)) - fix: checkpoint features format below v3,7 [#​2307](https://togithub.com/delta-io/delta-rs/pull/2307) ([ion-elgreco](https://togithub.com/ion-elgreco)) - fix: schema evolution not coercing with large arrow types [#​2305](https://togithub.com/delta-io/delta-rs/pull/2305) ([aersam](https://togithub.com/aersam)) - fix: clean up some non-datafusion builds [#​2303](https://togithub.com/delta-io/delta-rs/pull/2303) ([rtyler](https://togithub.com/rtyler)) - docs: fix typo [#​2300](https://togithub.com/delta-io/delta-rs/pull/2300) ([LauH1987](https://togithub.com/LauH1987)) - docs: make replaceWhere example compile [#​2299](https://togithub.com/delta-io/delta-rs/pull/2299) ([LauH1987](https://togithub.com/LauH1987)) - fix(rust): add missing chrono-tz feature [#​2295](https://togithub.com/delta-io/delta-rs/pull/2295) ([ion-elgreco](https://togithub.com/ion-elgreco)) - chore(python): bump to v0.16.1 [#​2294](https://togithub.com/delta-io/delta-rs/pull/2294) ([ion-elgreco](https://togithub.com/ion-elgreco)) - fix(rust): features not maintained in protocol after checkpoint [#​2293](https://togithub.com/delta-io/delta-rs/pull/2293) ([ion-elgreco](https://togithub.com/ion-elgreco)) - fix: merge predicate for concurrent writes [#​2291](https://togithub.com/delta-io/delta-rs/pull/2291) ([JonasDev1](https://togithub.com/JonasDev1)) - fix: replace assert and AssertionError with appropriate exceptions [#​2286](https://togithub.com/delta-io/delta-rs/pull/2286) ([joe-sharman](https://togithub.com/joe-sharman)) - docs: fix typo in delta-lake-polars.md [#​2285](https://togithub.com/delta-io/delta-rs/pull/2285) ([vladdoster](https://togithub.com/vladdoster)) - fix(python, rust): prevent table scan returning large arrow dtypes [#​2274](https://togithub.com/delta-io/delta-rs/pull/2274) ([ion-elgreco](https://togithub.com/ion-elgreco)) - fix(python): always encapsulate column names in backticks in \_all functions [#​2271](https://togithub.com/delta-io/delta-rs/pull/2271) ([ion-elgreco](https://togithub.com/ion-elgreco)) - fix(rust): read only checkpoints that match \_last_checkpoint version [#​2270](https://togithub.com/delta-io/delta-rs/pull/2270) ([ion-elgreco](https://togithub.com/ion-elgreco)) - fix: add .venv to .gitignore [#​2268](https://togithub.com/delta-io/delta-rs/pull/2268) ([gacharya](https://togithub.com/gacharya)) - feat(python, rust): add `set table properties` operation [#​2264](https://togithub.com/delta-io/delta-rs/pull/2264) ([ion-elgreco](https://togithub.com/ion-elgreco)) - docs: use dagster deltalake polars library [#​2263](https://togithub.com/delta-io/delta-rs/pull/2263) ([avriiil](https://togithub.com/avriiil)) - docs: update comment about r2 requiring locks [#​2261](https://togithub.com/delta-io/delta-rs/pull/2261) ([cmackenzie1](https://togithub.com/cmackenzie1)) - fix([#​2256](https://togithub.com/delta-io/delta.rs/issues/2256)): use consistent units of time [#​2260](https://togithub.com/delta-io/delta-rs/pull/2260) ([cmackenzie1](https://togithub.com/cmackenzie1)) - chore: update the changelog for rust-v0.17.1 [#​2259](https://togithub.com/delta-io/delta-rs/pull/2259) ([rtyler](https://togithub.com/rtyler)) - feat(python): release GIL in the write_deltalake function [#​2257](https://togithub.com/delta-io/delta-rs/pull/2257) ([franz101](https://togithub.com/franz101)) - chore(rust): bump datafusion to 36 [#​2249](https://togithub.com/delta-io/delta-rs/pull/2249) ([universalmind303](https://togithub.com/universalmind303)) - chore!: replace rusoto with AWS SDK [#​2243](https://togithub.com/delta-io/delta-rs/pull/2243) ([mightyshazam](https://togithub.com/mightyshazam)) - fix: handle conflict checking in optimize correctly [#​2208](https://togithub.com/delta-io/delta-rs/pull/2208) ([emcake](https://togithub.com/emcake)) - feat: logical Node for find files [#​2194](https://togithub.com/delta-io/delta-rs/pull/2194) ([hntd187](https://togithub.com/hntd187))

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Enabled.

♻ Rebasing: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

This PR was generated by Mend Renovate. View the repository job log.