When reconstructing the staging DB, it is necessary to produce an ORC file for the planet history as well as to update a changeset ORC file. The latter was covered by a make rule, but the former task was previously left to a manual process. This PR adds automation to help with this, and combines it with the changeset ORC update. Now, from ./deployment/batch/makefile, make prepare-bulk-ingest will create a cluster and add steps for both the planet history ORC conversion and the changeset ORC update. The cluster will terminate automatically after the steps terminate.
This PR replaces #200 after that PR was soiled by the commit of private information and the tree was never the same.
When reconstructing the staging DB, it is necessary to produce an ORC file for the planet history as well as to update a changeset ORC file. The latter was covered by a make rule, but the former task was previously left to a manual process. This PR adds automation to help with this, and combines it with the changeset ORC update. Now, from
./deployment/batch/makefile,make prepare-bulk-ingestwill create a cluster and add steps for both the planet history ORC conversion and the changeset ORC update. The cluster will terminate automatically after the steps terminate.This PR replaces #200 after that PR was soiled by the commit of private information and the tree was never the same.