latticexyz / mud

MUD is a framework for building autonomous worlds
https://mud.dev
MIT License
750 stars 187 forks source link

consider including field index (and field bytes offset) in Store events #2222

Closed holic closed 8 months ago

holic commented 9 months ago

Store's splice operations on static and dynamic fields use a field index to map to contract storage slots. Without this data expressed in events, offchain clients/indexers are ~forced to model the data as a single bytes blob for static field data and another for dynamic field data.

Since we already have access to the field index in these methods (and the field-specific bytes offset for dynamic fields), we could expose these as part of the event to allow for more flexible ways of modeling data offchain.

We should probably measure the gas of this to make sure it's worth the trade-off. Adding this also means we'd cement the "one splice event = one field update", which seems fine.

holic commented 9 months ago

Unclear how to handle the set event - I think we'd have to send down the whole field layout?

yonadaaa commented 9 months ago

I think "one splice event = one field update" is fine, I can't see why we'd splice many fields but not the whole record.

How do we feel then about removing start for the static data event? There's a disconnect that events operate over bytes when the codebase generally uses fields (StoreCore.setField, Number.getValue...).

I guess it would prevent us from building "schemaless" indexers, but table schemas are a Store feature after all - we shouldn't obscure them (in contrast to eg. World namespaces).

holic commented 9 months ago

The idea here is that we currently support creating schemaless indexers, but with the constraint that data is stored in two fields: static data and dynamic data.

However, that's not how it's actually stored onchain and I want to explore what additional args we could provide in events to offer indexer developers to have the option to store in a single blob for all like data vs. blob per field without a schema lookup.

You could still build this with the events we have now, but would require an additional schema lookup.

dk1a commented 9 months ago

For more schemalessness the indexer devs could use FieldLayout's total static length It's like how onchain the data is also schemaless and only FieldLayout is used,

it's still a lookup, but not strictly a schema lookup

How do we feel then about removing start for the static data event?

Some thoughts on making start an encapsulated implementation detail, for the more schemaful route:

We can replace start with startElementIndex or the like, without losing functionality, all while making table libs do less translation, but then splice methods in StoreCore would need to get FieldLayout, to get the necessary byte lengths. So the cost would be 1 more storage read I guess (or 1 more arg, if we think just sending it is fine)

holic commented 9 months ago
yonadaaa commented 8 months ago