swarm-game / swarm

Resource gathering + programming game
Other
834 stars 52 forks source link

Reuse environment definitions in `ToJSON` and `FromJSON` #2107

Open xsebek opened 1 month ago

xsebek commented 1 month ago

Is your feature request related to a problem? Please describe.

TLDR: the problem is that _envVars inside _envVars cause exponential JSON size.

Take this simple example:

def m2 = m; m end
def m4 = m2; m2 end
def m8 = m4; m4 end
def m16 = m8; m8 end
def m32 = m16; m16 end
And observe the growth of JSON output: empty m2 m4 m8 m16 m32 m1024
5 27 78 180 384 792 52200

This makes it impossible to get to other useful parts of robot JSON, like the log, if the program has enough definitions:

Describe the solution you'd like

The definitions should be reused - the inner references should link ({"link": "m8"}) to outer definition.

If we can check that the definitions are the same, we could prune them from inner scope:

- deriving instance FromJSON Env
- deriving instance ToJSON Env
+ instance FromJSONE Env Env
+ instance ToJSON (Env, Env)  -- or some newtype WithEnv

Describe alternatives you've considered

The current derived instance is broken, so maybe we could only keep the top environment, or none at all.

byorgey commented 1 month ago

This may actually be a good application for the (heretofore mythical) ToJSONE. I just want to make sure we avoid doing $O(n^2)$ work to repeatedly check the same definitions for equality.

xsebek commented 1 month ago

I think even $O(n^2)$ would be OK, since it would remove exponentially many inner references.

Outputting JSON is not done on main thread, so it would not freeze the game.

byorgey commented 1 month ago

Good point. Yes, we should not succumb to premature optimization here. Let's start with the simplest thing that works and we can optimize it later if necessary.