HydrologicEngineeringCenter / Vortex

data processing utilities
MIT License
28 stars 7 forks source link

Implement TemporalDataReader #116

Closed nickvan612 closed 1 month ago

nickvan612 commented 1 month ago

Add TemporalDataReader and TemporalDataReaderTest to read data and calculate as appropriate for the desired time interval. Add DataReader::getTimeRecords. Add BufferedDataReader and BufferedDataReaderTest to read data into a buffer, instead of the entire data. Update DssDataReader to cache the catalog, and use it to map between index and pathname. Migrate getUnitsString and getGridInfo from DssDataWriter to DssUtil. Add FilenameUtil::endsWithExtensions to replace regex pattern matching for file extensions to improve performance. Add Grid::toBuilder to make a similar copy of a Grid. Split NetcdfDataReader grid-set logic into GridDatasetReader. Add IndexMap to keep track of objects' indices. Add RecordIndexQuery to query a time record and return its index. Add VortexTimeRecord to store data's start and end time, and it implements Interval to work with RecordIndexQuery. Add MemoryDynamicWriteDataBuffer to buffer data to write based on available memory -- clears and processes buffer when memory cap is reached. Update NetcdfDataWriter to make it more robust. Update SerialBatchImporter to update progress properly. Update TimeConverter to cache formatter to improve performance. Add VortexTimeUtils to deal with time from a base time. Update VortexVariable to be more robust. Update WktFactory and WktParser to take into account AlbersEqualAreaEllipse.