Currently, we can reuse a module while overriding its params and initialBlocks.
However, there's no easy way to declare a completely generic module (ex: a mapper that converts a proto:sf.substreams.sink.database.v1.DatabaseChanges to type: proto:sf.substreams.entity.v1.EntityChanges) and use it on the output on our module, because we cannot override its "input".
The idea is to be able to swap a module's input with another input that has the same type.
in here would make it use the code from the imported module, but rewire the inputs and outputs, making sure that they are of the same type.
The converter author has a substreams.yaml like this:
specVersion: v0.1.0
package:
name: converter
version: v1.0.2
modules:
...
- name: example_dbout # write some code here that does nothing, just to link things.
kind: map
inputs:
source: sf.substreams.v1.Clock
output:
type: proto:sf.substreams.sink.database.v1.DatabaseChanges
- name: dbout_to_graphout
kind: map
inputs:
- map: example_dbout
output:
type: proto:sf.substreams.entity.v1.EntityChanges
TODO
The substreams client will do all the required remapping for that feature.
[x] Implement the use attribute in the yaml parser
[x] make sure that a module in substreams.yaml that has use can only specify (override) the following attributes:
inputs,initialBlockand the (not yet implemented)blockIndex(the params can always be set in theparams` section of the manifest)
[x] Also validate that the overridden inputs are similar to the previous one (same number, same types)
[x] Implement all the input rewire logic so that the generated modules are tweaked as requested
[x] Document the usage (with the example in this issue)
Context
Currently, we can reuse a module while overriding its params and initialBlocks.
However, there's no easy way to declare a completely generic module (ex: a mapper that converts a
proto:sf.substreams.sink.database.v1.DatabaseChanges
totype: proto:sf.substreams.entity.v1.EntityChanges
) and use it on the output on our module, because we cannot override its "input".The idea is to be able to swap a module's input with another input that has the same type.
ex
The line:
in here would make it use the code from the imported module, but rewire the inputs and outputs, making sure that they are of the same type.
The converter author has a substreams.yaml like this:
TODO
The substreams client will do all the required remapping for that feature.
use
attribute in the yaml parseruse
can only specify (override) the following attributes:inputs,
initialBlockand the (not yet implemented)
blockIndex(the params can always be set in the
params` section of the manifest)