opendatahub-io-contrib / datamesh-platform

Apache License 2.0
4 stars 3 forks source link

Document data insertion through Spark and data query process with code example #17

Closed caldeirav closed 1 month ago

caldeirav commented 2 months ago

Revisit the data product to leverage Spark-based insertion

@jpaulrajredhat to provide an example to @caldeirav to introduce into the data product

caldeirav commented 2 months ago

Note the possibility of direct ingestion with Iceberg FileIO / Spark: https://blog.min.io/lakehouse-architecture-iceberg-minio/