polarsignals / frostdb

❄️ Coolest database around 🧊 Embeddable column database written in Go.
Apache License 2.0
1.29k stars 65 forks source link

An exception that generates an empty parquet file #860

Closed jicanghaixb closed 4 months ago

jicanghaixb commented 4 months ago

If frostdb does not inject data and then closes frost db normally, it will generate an empty parquet file. Because close frostdb will write active block,but active block is null,so generate an empty parquet file,blow logs:

Details

timestamp=2024-05-09T11:34:00.051+0800 level=info logger=smart caller=frostdb/db.go:1004 message="closing DB" component=frostdb timestamp=2024-05-09T11:34:00.052+0800 level=error logger=smart caller=frostdb/table.go:514 message="failed to persist block" component=frostdb timestamp=2024-05-09T11:34:00.052+0800 level=error logger=smart caller=frostdb/table.go:515 message="failed to upload block reading magic header of parquet file: EOF" component=frostdb

Details

-rw-rw-r-- 1 0 5月 9 10:37 01HXDKDDGH00S0NG9NPA5J13H0.parquet -rw-rw-r-- 1 0 5月 9 10:40 01HXDKHYEV95AYPSFPATATEABD.parquet -rw-rw-r-- 1 0 5月 9 10:43 01HXDKRAC4JPK7PSH84HA21SZS.parquet

Although Iceberg's Maintenance can clean up these useless files, it is best to check when writing parquet files to avoid the generation of useless files and logs