hobuinc / silvimetric

Apache License 2.0
8 stars 3 forks source link

Kmann/manage #78

Closed kylemann16 closed 5 months ago

kylemann16 commented 5 months ago

Adding commands to manage the silvimetric tiledb database. These commands include

This PR reworks a couple of components:

  1. Shatter processes now operate in a distinct time window associated with their process number, which is an incrementing number produced from storage
  2. We now consolidate the fragments that are written for a shatter process down into one fragment, for ease of time traveling as well as ease of use.
  3. Storage class now includes some more wrapper functions for tiledb including
    • delete which deletes fragments associated with a shatter process number (timestamp)
    • get_fragments_by_time which grabs tiledb fragments and filters them by timestamp
    • consolidate_shatter which consolidates by a timestamp
    • mbrs which grabs the minimum bounding rectangle for a list of fragments filtered by timestamp

This also includes the new test tile test_commands.py which will test commands that don't need a full test suite including info, delete, restart, and resume