search

Eventual-Inc / Daft

Distributed DataFrame for Python designed for the cloud, powered by Rust
https://getdaft.io
Apache License 2.0
1.76k stars 105 forks source link

[FEAT] Enable anonymous S3 access for Delta #2206

Closed jaychia closed 2 weeks ago

jaychia commented 2 weeks ago

Enables anonymous mode for deltalake reading

Verified that this works:

df = daft.read_delta_lake("s3://daft-public-data/nyc-taxi-dataset-2023-jan-deltalake/", io_config=daft.io.IOConfig(s3=daft.io.S3Config(anonymous=True, region_name="us-west-2")))
codecov[bot] commented 2 weeks ago

Codecov Report

Attention: Patch coverage is 0% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 85.11%. Comparing base (a0fd6ec) to head (7a909a7). Report is 3 commits behind head on main.

Additional details and impacted files [![Impacted file tree graph](https://app.codecov.io/gh/Eventual-Inc/Daft/pull/2206/graphs/tree.svg?width=650&height=150&src=pr&token=J430QVFE89&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Eventual-Inc)](https://app.codecov.io/gh/Eventual-Inc/Daft/pull/2206?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Eventual-Inc) ```diff @@ Coverage Diff @@ ## main #2206 +/- ## ========================================== + Coverage 85.10% 85.11% +0.01% ========================================== Files 68 68 Lines 7391 7391 ========================================== + Hits 6290 6291 +1 + Misses 1101 1100 -1 ``` | [Files](https://app.codecov.io/gh/Eventual-Inc/Daft/pull/2206?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Eventual-Inc) | Coverage Δ | | |---|---|---| | [daft/delta\_lake/delta\_lake\_scan.py](https://app.codecov.io/gh/Eventual-Inc/Daft/pull/2206?src=pr&el=tree&filepath=daft%2Fdelta_lake%2Fdelta_lake_scan.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Eventual-Inc#diff-ZGFmdC9kZWx0YV9sYWtlL2RlbHRhX2xha2Vfc2Nhbi5weQ==) | `86.66% <0.00%> (ø)` | | ... and [1 file with indirect coverage changes](https://app.codecov.io/gh/Eventual-Inc/Daft/pull/2206/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Eventual-Inc)