Indexers factories

[fracek]

Is your feature request related to a problem? Please describe.

In some cases (notably: DEXes) the set of smart contracts to index is not known at deployment, but changes dynamically over time.

Describe the solution you'd like

We should have a special "factory" sink type that is used to dynamically deploy new child indexers.

The following example listens for contract deployments, then for each contract it starts 2 indexers (one to index swaps, one to track user balances).

export const config = {
  // ... normal config
  sinkType: "factory",
  sinkOptions: {
    // no options?
  }
}

export default function transform({ header, events }: Block) {
  const startingBlock = header.blockNumber;
  return events.flatMap(({ event }) => {
    const contractAddress = event.data[0];
    // return the sinks config
    return [{
      id: `swaps-${contractAddress}`,
      src: "src/swaps.ts",
      env: {
        "STARTING_BLOCK": startingBlock,
        "CONTRACT_ADDRESS": contractAddress,
      }
    }, {
      id: `balances-${contractAddress}`,
      src: "src/balances.ts",
      env: {
       "STARTING_BLOCK": startingBlock,
        "CONTRACT_ADDRESS": contractAddress,
      }
    }]
  })
}

The sub indexers should run independently (so data is eventually consistent).

The challenge is how to store which indexers have been deployed so that they can be restarted on restart. In theory it only needs to store the indexer source path and the environment variables used to start. Maybe it makes sense to re-use the persistence layer (none, fs, etcd) to also store this information?

The idea is that using this type of indexers, users are able to build "indexers trees" like the following:

pool-factory/
├── dai-usdc-balances
├── dai-usdc-swaps
├── eth-dai-balances
├── eth-dai-swaps
├── eth-usdc-balances
└── eth-usdc-swaps

Additional Context

This one can share a lot of code with #226

[fracek]

Factories are now supported.