Closed mkchungs closed 1 year ago
Consistent naming convention across: local/gs.
const step0_createRec = 0 function: cpblk(dt, chainID) const step1_cpblk = 1 function: cpblk(dt, chainID) Source: crypto_ethereum.. Output: gs://evm_etl/YYYY/MM/DD/chainID/ const step2_backfill = 2 function: backfill(dt, chainID) Source: gs://evm_etl/YYYY/MM/DD/chainID/ LocalTmp: /tmp/evm_etl/YYYY/MM/DD/chainID/ Output: cbt evmchain${chainID} const step3_indexEvmChain = 3 function: index_evmchain(chainID, dt) Source: cbt evmchain${chainID} LocalTmp: /tmp/evm_decoded/YYYY/MM/DD/chainID/tableId* const STEP4_cpEvmDecodedToGS = 4 function: cpEvmDecodedToGS(dt, chainID, dryrun) Input: /tmp/evm_decoded/YYYY/MM/DD/chainID/tableId.. Output: gs://evm_decoded/YYYY/MM/DD/tableId../chainID.json const STEP5_loadGSEvmDecoded = 5 function: loadGSEvmDecoded(dt, chainID) Input:gs://evm_decoded/YYYY/MM/DD/ localInput: /disk1/evmschema
NOTE: local indexing must use /YYYY/MM/DD/chainID/ to prevent concurrent indexing issues
Consistent naming convention across: local/gs.
NOTE: local indexing must use /YYYY/MM/DD/chainID/ to prevent concurrent indexing issues