To allow augmented diffs to be stored and listed more efficiently, AugmentedDiffSource should read from the partitioned, compressed versions of the sequences. Once this is done, the secondary write path in overpass-diff-publisher can be removed and existing, uncompressed diffs removed.
AugmentedDiffSource
currently assumes that augmented diffs (produced by https://github.com/mojodna/overpass-diff-publisher) are available by sequence number as uncompressed GeoJSON in the root of the target. This is currently implemented as https://github.com/azavea/osmesa/blob/4a1f3fd9c22cc02172805729b91036f35d66bfd8/src/common/src/main/scala/osmesa/common/sources/AugmentedDiffSource.scala#L30-L67https://github.com/mojodna/overpass-diff-publisher/commit/309043fd9a86235c7ce745bcc60e003f35ffb762 and https://github.com/mojodna/overpass-diff-publisher/commit/911a3c7ef514f6fcd30d7545549afd4c85612776 changed overpass-diff-publisher's behavior so that it writes gzip-compressed data to partitioned paths (i.e.
000/000/000.json.gz
instead of0.json
). (It currently writes to both keys for compatibility.)To allow augmented diffs to be stored and listed more efficiently,
AugmentedDiffSource
should read from the partitioned, compressed versions of the sequences. Once this is done, the secondary write path in overpass-diff-publisher can be removed and existing, uncompressed diffs removed.