Open tanejagagan opened 10 months ago
This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.
Currently Spark Structured streaming writes metadata files inside _spark_metadata directory with following content in json format {code} { path : String, size : Long, isDir : Boolean, modificationTime : Long, blockReplication: Int, blockSize: Long, action : String } {code} IceBerg should provide a utility to generate _iceberg_metadata files based on the content present in _spark_metadata directory. This will also help preserve the snapshots of the given table