opera-adt / COMPASS

COregistered Multi-temPorAl Sar Slc
Apache License 2.0
39 stars 18 forks source link

geocode stack adjustments for a gzipped-json burst db #97

Closed scottstanie closed 1 year ago

scottstanie commented 1 year ago

This PR has a few changes that came up while testing out the COMPASS wrapper workflow https://github.com/opera-adt/sweets

The main changes are to allow a format of the burst db that's a simple JSON file instead of a sqlite database. Since it takes <1sec to load 1M rows into memory, there's little noticeable difference in the s1_geocode_stack runtime. The nice part is the DB is only ~15 MB when it's a gzipped JSON file.

Something that should be sorted out before this is merged is the burst_db repo, where this was implemented https://github.com/opera-adt/burst_db/compare/main...scottstanie:burst_db:test-json-gz But really we can store the v1 version of the DB somewhere easy to download because it's so small.

scottstanie commented 1 year ago

Also, if it's helpful to separate discussion between adding the ability to run on .SAFE files and using a .json.gz format, I can close this PR and break them out.

vbrancat commented 1 year ago

Breaking those discussions in two separate PRs would be extremely helpful (more work on you though, :( )

scottstanie commented 1 year ago

Closing in favor of separate incoming PRs