EarthScope / mseed3-utils

Validator and other utilities for the xSEED data format
Apache License 2.0
8 stars 2 forks source link

Check payload from read data, instead of re-reading #14

Closed chad-earthscope closed 5 years ago

chad-earthscope commented 5 years ago

Currently the payload checks are performed by re-reading and re-parsing the entire file being checked (in check_file.c), i.e. all files are read twice.

This should be improved by performing the payload check for each record while it is already read into memory, in the range noted by:

//TODO check payload via buffer, for now payloads are checked via libmseed after all headers are checked
...
//TODO validate payload using the buffer contains
chad-earthscope commented 5 years ago

Previous program flow: 1) Read all records from file, check headers, identifier and extra headers 1) Read all records from file (again), decompressing all data 1) Decompress each record (again) to determine payload viability

Clearly this is not optimal. There seems to have been a change in strategy from doing raw validation to doing validation using libmseed, but the work was not streamlined for this new approach.