Closed dimitrovmaksim closed 10 months ago
@dimitrovmaksim thanks for this, I think Matchstick may also need updates to support File Data Sources?
@dimitrovmaksim thanks for this, I think Matchstick may also need updates to support File Data Sources?
Yeah, File Data Sources are not currently supported as well
Btw @azf20 just FYI, it seems some changes that were made to the graph-node in order to re-use some modules/functionalities in matchstick have been reverted/moved/removed, so updating the graph-node dependencies to the latest version in matchstick could require a PR to the graph-node as well.
Progress:
Main scope
Extended scope
The Issue
Currently there are two new graph-node functionalities
loadInBlock
andloadRelated
which are not implemented/mocked in Matchstick. This causes matchstick to panic and it prevents some users from properly testing their subgraphs.loadInBlock
Description:
From my understanding
loadInBlock
tries to load an entity from some kind of a cache that persists for the time a specific block is being indexed. The entity will be present in the cache only if it was created or updated in that block.For example event
A
occurs in block 123, in our handler we create an entityB
with ID the address (0x0...1) of the emitter contractC
. Later in that block another eventA
is emitted by contractC
. If we try to load the entity usingB.loadInBlock("0x0...1")
we should get the entity we have created during the first occurrence of the eventA
.But if in block 124, contract
C
has once again emitted eventA
and we try to load the entity usingB.loadInBlock("0x0...1")
, the function will returnnull
because the entity is no more in the cache and instead we should try to load it from the database.Matchstick implementation:
My proposed solution would be to:
loadInBlock
behaviour to returnnull
loadRelated
Description:
loadRelated
is a function that allows users to load derived entities into their handlers. This was previously not possible in thegraph-node
, but was possible with Matchstick due a bug in the graph-cli that generated getter (and setter) functions for those fields and the way the Matchstick's own entity store works. Derived fields are virtual fields and do not actually exists in thegraph-node
database. Instead they are created during query time and are only accessible when querying the subgraph. More info on derived fields https://thegraph.com/docs/en/developing/creating-a-subgraph/#reverse-lookupsMatchstick's own store
Matchstick does not work with a database, but implements its own key/value entity store. In the Matchstick store the derive fields actually exists and they keep the IDs of the child entities, hence why the getter/setter functions worked in Matchstick. While this was helpful for users to test their relations, it caused a lot of issues (especially with handling both String and Byte IDs) and made the code complex. It will probably be wise to refactor the store and make it function similarly to the graph-node, i.e dynamically construct the derived fields when requested. We already keep the derived relations in a separate derived key/value store, but it may not be needed anymore, because the child entity's name, the parent id and the derivedFrom field should come from the
loadRelated
function itself, which should make it easier to find it in our entity store.Matchstick implementation:
Entity Store:
Looking into the
function loadRelated(entity: string, id: string, field: string): Array<Entity>;
function we may be able to:derived
mapping. But the function itself should provide us with the relation info we need in order to fetch the child entities from the entities store, so this mapping becomes obsolete as well as this whole module.logStore()
:The potential change to the store will affect what the
logStore()
function prints to the console. There's several ways to approach it:logStore()
logEntity(entityId)
function that will only print the specified entity and it's relationslogStore()
that if set to true will also construct and print the relationslogStore()
to accept a set of options{related: true, entityId: 123}
that will combine 2. and 3.Update Matchstick dependencies with the latest graph-node version:
Description:
Matchstick depends on some graph-node functionalities. It hasn't been updated for a while, so will need to do that in Cargo.toml. This may lead to issues that will need to be fixed
Resources:
Matchstick store and derived_fields implementation:
Entity store - https://github.com/LimeChain/matchstick/blob/fix-derived-with-byte-ids/src/context/mod.rs#L96 Derived relations store - https://github.com/LimeChain/matchstick/blob/fix-derived-with-byte-ids/src/context/mod.rs#L115 Initialising the relations store - https://github.com/LimeChain/matchstick/blob/fix-derived-with-byte-ids/src/context/derived_schema.rs Save Entity to store - https://github.com/LimeChain/matchstick/blob/fix-derived-with-byte-ids/src/context/mod.rs#L503 Saving the derived fields 🤯 - https://github.com/LimeChain/matchstick/blob/fix-derived-with-byte-ids/src/context/derived_fields.rs Get entity from store - https://github.com/LimeChain/matchstick/blob/fix-derived-with-byte-ids/src/context/mod.rs#L478 logStore - https://github.com/LimeChain/matchstick/blob/fix-derived-with-byte-ids/src/context/mod.rs#L181 clearStore - https://github.com/LimeChain/matchstick/blob/fix-derived-with-byte-ids/src/context/mod.rs#L190
Matchstick-as
https://github.com/LimeChain/matchstick-as
Matchstick docs
https://github.com/LimeChain/matchstick https://thegraph.com/docs/en/developing/unit-testing-framework/
Demo-subgraph
https://github.com/LimeChain/demo-subgraph