search

ARPA-SIMC / arkimet

A set of tools to organize, archive and distribute data files.
Other
15 stars 5 forks source link

arki-check with error type dataset #299

Open brancomat opened 1 year ago

brancomat commented 1 year ago

repacking an error type dataset results in an error (*):

$ arki-check error/
error:2019/03-23.grib: should be rescanned
error:2019/03-24.grib: should be rescanned
error: check 0 files ok, 2 files should be rescanned
$ arki-check -f error/
error:2019/03-23.grib: rescanned
error:2019/03-24.grib: rescanned
error: check 0 files ok, 2 files rescanned
$ arki-check -f -r error/
error: repack: running VACUUM ANALYZE on the dataset index, if applicable
error: repack 2 files ok

example dataset: error.tar.gz

this was discovered by accident by @lidiabressan and I don't see many real word cases in which it makes sense to launch a repack of an error dataset but it's complicated: on one side it could contain heterogeneous data (bufr + grib + odimh5) making a repack a bit tricky, on the other the doc states that the delete age field is applicable to error type dataset.

spanezz commented 1 year ago

I think I cannot see an error in the output above. I tried reproducing it, and I cannot see an error here either, and arki-check exits successfully with 0.

The only thing I can think at the moment is: could it be that the fact that arkimet repeats the dataset name before each line in the output is making it look like it's printing error messages?