Client syncing does not scale

[pwrstudio]

Problem

Current syncing of indexer state to client does not scale.

Getting data from the indexer is quick and no problem. It is the updating of the entities store that takes long (currently about 10 seconds) and blocks the UI in the process, so we can't show any progress indication either.

We are currently getting everything from the indexer, syncing it all to the entitites store, then filter it down to only the machines etc... that are in the player's own pod.

This problem will get worse and worse very quickly.

Current method

We set up listeners for all tables we are interested in here: https://github.com/Moving-Castles/this-cursed-machine/blob/016d5f922cdb0d997f174a7f77b7136a43ff4c60/packages/client/src/svelte/initNetwork.ts#L25

The svelte store is then update on each update from the chain: https://github.com/Moving-Castles/this-cursed-machine/blob/016d5f922cdb0d997f174a7f77b7136a43ff4c60/packages/client/src/svelte/modules/systems/createComponentSystem.ts#L20

So far this works OK once you are in the game – but not on load when you get 3000+ updates quickly.

Possible solution 1

Filter what we sync from the indexer.

We need:

• All orders, recipes, offers, materials.
• gameConfig
• all players names
• only machines, tanks that are in the player's pod (same carriedBy component value)

Can we filter by the carriedBy value?

How do we know the id of the players pod before syncing?

Possible solution 2

Get state from indexer. Set entities in one go. Avoiding the many calls to entities.update.

Possible solution 3

Move updating the entities store to a webworker.

Probably not possible.