Snapshotting, freezing or cloning the World

[cjay]

I think it would be very useful to be able to create read-only snapshots of the World state. For example when running different threads for graphics and game logic, the game logic could put snapshots of the World into an MVar to give the graphics thread access to a consistent state of the world whenever it starts a new frame. There would be no need to stop the game logic while talking to the graphics API.

Snapshots could also be useful for savegames, replays and other things. Maybe even a subset of the components could be specified when snapshotting to avoid retaining unwanted data.

I've noticed that most of the Stores are implemented as an IORef with a pure value inside, so I suppose the snapshots could mostly be made with neglegible computational overhead.

[jonascarpay]

This is an excellent question, and something that should be addressed in some piece of documentation, since everybody will run into this at one point or another. For now, I'll give my thoughts here.

In the past (not with apecs), I've done things like MyWorld (f :: * -> *). If f is Identity or Vector, it's an immutable copy of the game world, if f is IORef or IOVector, for example, it's mutable. In this language, you have a very clear vocabulary to move between the different representations:

freeze :: MyWorld IORef -> IO (MyWorld Identity)

In apecs, this wouldn't work as easily, since every Component can have a different type of underlying storage. It should be possible to e.g. give every Storage an associated FrozenStorage, maybe that's a good idea. I would love to see this as a separate package, but I'm not convinced it's a good idea to have first-class support for it.

My objection is that it's rarely useful to freeze the entire world. There's usually more efficient ways to freeze the parts of the world that you are interested in, and these tend to play better to apecs's strengths.

• Say you want to save the game. Just create a list of components you're interested in saving, encode and write the list, and to load just mapM_ add.
• Now, say you need to render, i.e. write pieces of data to GPU buffers. The simplest solution is to just cmapM the relevant components and write the buffer piecewise. If you really want to squeeze performance you could also use a Cache, getStore to get the underlying mutable vector, and then freeze and copy that vector directly. If you use a Storable-based Cache, there might even be zero-copy ways to do this.
• apecs-stm demonstrates how you can define stores that are thread-safe
• I'm not entirely sure how I'd do replays. You could do it the same way as above, just saving every frame. What would be cooler is to write a Store that tracks its own history, and allows indexing at different points in time.

In conclusion, my philosophy here is the flexibility to easily write your own perfect solution is more important than making the core of apecs more complicated for the sake of a solution might not be ideal. I'm interested in hearing people's thoughts on this.

[dpwiz]

I'd like to note sometimes you just can't "dump everything to file". One non-serializable component (like code inside components) will break the scheme.

[cjay]

My objection is that it's rarely useful to freeze the entire world. There's usually more efficient ways to freeze the parts of the world that you are interested in, and these tend to play better to apecs's strengths.

It seems we have very different philosophies colliding here.

On the one hand, if we think about the world as a mutable set of mutable objects, what you write makes sense. And on the surface, Apecs seems to embrace that imperative mindset.

From a FP perspective, if we think about the world as a immutable data structure, with every modification or time step resulting in a new world value, it makes a lot of sense to just take one of the world values and work with it. Sharing and garbage collection takes care of the rest, so it doesn't matter much if we are only interested in a part of the world. We don't pay anything for copying a world value, and we can reuse functions that work with world values everywhere.

With the FP approach, saving and rendering can both be done concurrently with mutating the world. If we have immutable data structures under the hood anyway, I think saying "there's more efficient ways to freeze parts of the world" is simply not true. Of cause it's different once there is at least one truly mutable storage. I'm not sure if STM is really capable of delivering the same amount of parallelism. It might depend on how fine grained the data structures are decomposed into STM variables, and how many things are going on that force transactions to retry.

I'm not experienced enough in gamedev to be sure, but I have the feeling that there is all kinds of stuff that relies on working with a temporally consistent worldview, which would require stopping the world mutation.

Anyway, maybe FrozenStorage is the easy answer to all this. Is it possible to somehow put a bunch of Storages that were retrieved with getStore together to treat them like a World, cmap over it etc.?

[DavidEichmann]

Just had a look at apecs and I'm quite impressed! I'm trying to get a pure implementation of a game loop going where I can freely rollback and replay inputs. This will only be safe if the world state is immutable! So I want to use ST as my underlying monad, but this doesn't seem possible! MonadIO constraints are littered through the code base. Indeed apecs uses IORef in the store types :-(. I may have some free time to give it a go, but is there some fundamental reason we cant get rid of the MonadIO constraint? I would expect that we mostly just need to use STRef in place of IORef.

[DavidEichmann]

Ah I've just seen PR #72. Having a pure world data type sounds good to me :-)