-
### Skill Name
Spark, delta lake, airflow
### Why?
its popular with data engineer...
other skills are airflow, delta lake... which also u can add..
### Reference Image
spark: https://commons.w…
-
### Is your feature request related to a problem or challenge?
I personally think making it easy to use DataFusion with the "open data lake" stack is very important over the next few months.
@jul…
alamb updated
14 hours ago
-
## Feature request
#### Which Delta project/connector is this regarding?
- [x] Spark
- [ ] Standalone
- [ ] Flink
- [ ] Kernel
- [ ] Other (fill in here)
### Overview
Implement bucke…
-
# Description
The docs should open by explaining what the basic use cases are for a lakehouse and delta-rs in particular. At present we have
This is the documentation for the native Rust/Python i…
-
## Bug
Currently, if delta.checkpoint.writeStatsAsStruct is set to true, the output contains parsed partition values but does not include parsed stats.
I think the [code](https://github.com/delta…
-
_As a Developer, I want to gather and utilize data from System Views in Microsoft Fabric to monitor system performance and query activity so that we can effectively score and tag stale content._
**Re…
-
# Iceberg
- Partitioning pruning support https://github.com/ClickHouse/ClickHouse/issues/52054 @divanik
- Time Travel Support: https://github.com/ClickHouse/ClickHouse/pull/71072 @divanik @bretth…
-
Hi,
I am trying to use an ADLS Gen2 SAS token to read and query a Delta table. I generated a folder-level SAS token. Now, I am attempting to access the Delta table using the following code snippet:
…
-
polars seems to support it but it's implemented on the python side: https://pola-rs.github.io/polars/py-polars/html/reference/api/polars.read_delta.html
the underlying delta lake interface lib is w…
-
## Bug
### Describe the problem
#### Steps to reproduce
1. Create a Test File (here using Databricks)
```python
spark.conf.set("spark.sql.parquet.outputTimestampType", "TIMESTAMP_MILLIS")
…