streamingfast / substreams

Powerful Blockchain streaming data engine, based on StreamingFast Firehose technology.
Apache License 2.0
159 stars 45 forks source link

Add Substreams Block Filtering to substreams.yaml and protobuf #410

Closed sduchesneau closed 6 months ago

sduchesneau commented 6 months ago
  1. A new module type, blockIndex that will be used to create the index used by the filter (extracting labels from transactions, etc.) (this type will have a special handler)
    modules:
    - name: my_cool_index
    type: blockIndex
    output:
    type: proto:sf.substreams.index.v1.Keys

    A module of this type does not have params in its input, initialBlock. The output type is forced to be sf.substreams.index.1.Keys (we can still move this around if need be, but it's a core construct of the engine, it could also be renamed Labels as @sduchesneau put it.).

message Keys {
  repeated string keys = 1;
}
  1. A new block inside module definitions, called blockFilter, which contains module (the reference to a blockIndex type of module) and query (a string that will be used to evaluate the blocks against the filter, see https://github.com/streamingfast/substreams/issues/409)
modules:
- name: my_cool_map
  blockFilter:
    module: eth_sieve:contracts_and_events
    query: (addr:USDC || addr:PANCAKE) && (evsig:Transfer || evsig:TransferFrom)
  1. A new top-level attribute, called blockFilters, which allows an override for "params":
    blockFilters:
    myModuleName: (addr:123123 || addr:234234) && (evsig:123123 || evsig:0x345)

    (this will be done client-side)

ArnaudBger commented 6 months ago

435