Partial world update by network

[Gidroshvandel]

Hello, I am trying to make a network game using fleks.

I only found the snapshot api for mass updating the world. But I don't want to transmit the whole world over the network. It would be convenient to be able to add entities received over the network with their components.

Could you tell me the best way to implement this by using fleks?

[Quillraven]

Hi,

Could you give me more details what you are trying to achieve? Maybe also a code example how you want your code to look like?

Because I am not 100% sure what your problem is. The snapshot functionality is meant for saving/loading an entire world. Typical use-case is a save and load system in a game.

Other entity creation while the game is running should be done with world.entity.

I normally have helper functions for different entity types. E.g. one function to create a NPC with it's necessary components. One function for a special effect entity, etc.. Those functions then just take a minimum amount of arguments like e.g. NPC would be XY coordinate and NPC type. The function then creates an entity with a position, sprite, animation, AI, ... component.

Maybe this helps you already? Otherwise, please provide more details about your issue and how you'd like to solve it.

If something is missing I am gladly adding it to the library 🙂

[Gidroshvandel]

For example i have class like this to track changes in the world

sealed class UpdateData() {
    class Changed(val entityListMutableMap: Map<Entity, List<Component<*>>>) : UpdateData()

    class Added(val entity: Entity, val list: List<Component<*>>) : UpdateData()

    class Removed(val entity: Entity) : UpdateData()
}

And i want to apply this changes (I'm assuming getting list: List<UpdateData> from the server)

fun applyUpdateData(list: List<UpdateData>) {
        list.forEach { updateData ->
            when (updateData) {
                is UpdateData.Added -> world.entity {
                    if (it.id != updateData.entity.id) { // I'm not sure that the entity will be created with the correct id
                        throw IllegalStateException()
                    } else {
                        updateData.list.forEach { component ->
                            it += component // Entity.plusAssign required reified T : Component<T> but my component is Component<*>
                        }
                    }
                }
                is UpdateData.Changed -> {
                    updateData.entityListMutableMap.forEach { entry ->
                        entry.key.configure {
                            entry.value.forEach { component ->
                                it += component // Entity.plusAssign required reified T : Component<T> but my component is Component<*>
                            }
                        }
                    }
                }
                is UpdateData.Removed -> updateData.entity.configure {
                    it.remove()
                }
            }
        }
    }

Perhaps I chose the wrong way to solve the problem, can you tell me the best way to do this?

[Gidroshvandel]

I am getting list of components by this method fun snapshotOf(entity: Entity): List<Component<*>>

And it is expected to have a method that applies changes to some entity Like this fun applyEntitySnapshot(entity: Entity, list: List<Component<*>>)

Or some other way to do this

[Quillraven]

Unfortunately I don't have any experience with network game development. I know at least that some games run the entire logic on the server and the clients only receive basic stuff to render the scene properly. But damage calculations, collision handling, etc. Is done on the server.

That way you would not need to track the changes and send it to the clients if I understand your current approach correctly.

However, as I said, I am not an expert on that and I think it won't harm if we add a new function to Fleks to allow to set a specific configuration of components for a specific entity. Similar to what loadSnaoshot does but for a single entity.

I will try to come up with something by tomorrow and let you know when you can test it.

I think then it is easier for you to judge if it helps or not 🙂

[jobe-m]

@Gidroshvandel How do you make sure that on each end only new entities are created which do not use an entity ID which might be in use on the other side? If you have game logic running on each side than both will create new entities on their own and try to sync it over the net with the other side. That will lead to a ID clash very fast.

[Gidroshvandel]

Unfortunately I don't have any experience with network game development. I know at least that some games run the entire logic on the server and the clients only receive basic stuff to render the scene properly. But damage calculations, collision handling, etc. Is done on the server.

That way you would not need to track the changes and send it to the clients if I understand your current approach correctly.

However, as I said, I am not an expert on that and I think it won't harm if we add a new function to Fleks to allow to set a specific configuration of components for a specific entity. Similar to what loadSnaoshot does but for a single entity.

I will try to come up with something by tomorrow and let you know when you can test it.

I think then it is easier for you to judge if it helps or not 🙂

This should help in this situation, thanks!

Unfortunately, I also do not have the proper experience in creating multiplayer games, I focus on this guide https://www.gabrielgambetta.com/client-server-game-architecture.html

I suppose that keeping all the logic only on the server will have too much latency

[Gidroshvandel]

@Gidroshvandel How do you make sure that on each end only new entities are created which do not use an entity ID which might be in use on the other side? If you have game logic running on each side than both will create new entities on their own and try to sync it over the net with the other side. That will lead to a ID clash very fast.

Yes, this presents a problem, and I don't have a known solution yet... I will probably have to create an additional entity identification system that will be managed by the server

Maybe something like artemis-odb solution https://github.com/junkdog/artemis-odb/issues/267

[Quillraven]

Hm, I don't like the solution in the linked thread. From what I understand, it is an "in between" mapping from one entity id to an entity id in the world and actually in both directions.

Not 100% sure how this master/slave worlds are working in reality but wouldn't it be better if master (=server) is responsible for unique entity ids? He could simply use the built in Fleks world ID creation for that.

And for the client worlds it might be useful if you can create entities with a specific ID, coming from the server. We could still use the normal Fleks implementation for that but I need to do some minor adjustments to make it work.

E.g. you can call world.entity(id = yourId) { ... }. Wouldn't that be a better solution?

[jobe-m]

@Gidroshvandel How do you make sure that on each end only new entities are created which do not use an entity ID which might be in use on the other side? If you have game logic running on each side than both will create new entities on their own and try to sync it over the net with the other side. That will lead to a ID clash very fast.

Yes, this presents a problem, and I don't have a known solution yet... I will probably have to create an additional entity identification system that will be managed by the server

Maybe something like artemis-odb solution junkdog/artemis-odb#267

@Quillraven I just had the idea that we could reserve a bunch of IDs for a world. So let's assume we create two worlds locally e.g. in a test program. Each world can create new entities with IDs from a list which is unique i.e. the IDs are exclusive for that world. Then it would be save if any world creates new entities that those entities when synced to the other world are not clashing. Thus we would need the feature to specify the range of IDs for a world at creation which this world is allowed to create. Other IDs can only get into that world from outside. Then we just need to make sure that both worlds use different ranges for their IDs.

When syncing entities from outside with other IDs then we need a function to specify the ID for the to be created entity.

I hope you got the point and that this works at all :)

[Gidroshvandel]

Hm, I don't like the solution in the linked thread. From what I understand, it is an "in between" mapping from one entity id to an entity id in the world and actually in both directions.

Not 100% sure how this master/slave worlds are working in reality but wouldn't it be better if master (=server) is responsible for unique entity ids? He could simply use the built in Fleks world ID creation for that.

And for the client worlds it might be useful if you can create entities with a specific ID, coming from the server. We could still use the normal Fleks implementation for that but I need to do some minor adjustments to make it work.

E.g. you can call world.entity(id = yourId) { ... }. Wouldn't that be a better solution?

This is certainly a simpler and more understandable solution, but it will only work if clients contact the server to create new entities. This carries a large delay, you need to send information to the server, wait for processing and get a response, which can be unacceptably long.

I'm trying to implement a more complex, but it seems to me more efficient scheme, which requires the client to be able to predict the actions of its user and coordinate them with the server. The server in this scheme only validates user data and synchronizes multiple clients.

I'm trying to organize a client-server game architecture which can be summarized as follows:

• Server gets inputs from all the clients, with timestamps
• Server processes inputs and updates world status
• Server sends regular world snapshots to all clients
• Client sends input and simulates their effects locally
• Client get world updates and
  • Syncs predicted state to authoritative state
  • Interpolates known past states for other entities

From a player’s point of view, this has two important consequences:

• Player sees himself in the present
• Player sees other entities in the past

Details can be seen in this series of articles: https://www.gabrielgambetta.com/entity-interpolation.html

[Gidroshvandel]

@Gidroshvandel How do you make sure that on each end only new entities are created which do not use an entity ID which might be in use on the other side? If you have game logic running on each side than both will create new entities on their own and try to sync it over the net with the other side. That will lead to a ID clash very fast.

Yes, this presents a problem, and I don't have a known solution yet... I will probably have to create an additional entity identification system that will be managed by the server Maybe something like artemis-odb solution junkdog/artemis-odb#267

@Quillraven I just had the idea that we could reserve a bunch of IDs for a world. So let's assume we create two worlds locally e.g. in a test program. Each world can create new entities with IDs from a list which is unique i.e. the IDs are exclusive for that world. Then it would be save if any world creates new entities that those entities when synced to the other world are not clashing. Thus we would need the feature to specify the range of IDs for a world at creation which this world is allowed to create. Other IDs can only get into that world from outside. Then we just need to make sure that both worlds use different ranges for their IDs.

When syncing entities from outside with other IDs then we need a function to specify the ID for the to be created entity.

I hope you got the point and that this works at all :)

This seems to solve the problem if the limit of entities in the world is known in advance. The server can easily give it to clients when it connects because it knows the number of clients and the ranges of each client.

But what happens if one of the clients exceeds the limit? This seems to be a rather inconvenient limitation for large games. In my case, this solution should work, but it does not look flexible

[Quillraven]

Hm, I don't like the solution in the linked thread. From what I understand, it is an "in between" mapping from one entity id to an entity id in the world and actually in both directions. Not 100% sure how this master/slave worlds are working in reality but wouldn't it be better if master (=server) is responsible for unique entity ids? He could simply use the built in Fleks world ID creation for that. And for the client worlds it might be useful if you can create entities with a specific ID, coming from the server. We could still use the normal Fleks implementation for that but I need to do some minor adjustments to make it work. E.g. you can call world.entity(id = yourId) { ... }. Wouldn't that be a better solution?

This is certainly a simpler and more understandable solution, but it will only work if clients contact the server to create new entities. This carries a large delay, you need to send information to the server, wait for processing and get a response, which can be unacceptably long.

I'm trying to implement a more complex, but it seems to me more efficient scheme, which requires the client to be able to predict the actions of its user and coordinate them with the server. The server in this scheme only validates user data and synchronizes multiple clients.

I'm trying to organize a client-server game architecture which can be summarized as follows:

• Server gets inputs from all the clients, with timestamps
• Server processes inputs and updates world status
• Server sends regular world snapshots to all clients
• Client sends input and simulates their effects locally
• Client get world updates and
  • Syncs predicted state to authoritative state
  • Interpolates known past states for other entities

From a player’s point of view, this has two important consequences:

• Player sees himself in the present
• Player sees other entities in the past

Details can be seen in this series of articles: https://www.gabrielgambetta.com/entity-interpolation.html

Can you give an example on how that will look like? The way I imagine it with 2 clients and one server:

• Client A has its own local world
• Client B has its own local world
• Server has no world
• Client A creates an entity with ID 0
• Client B creates an entity with ID 0
• The server now synchronizes both of these worlds and sends updates: -- Client B gets the entity information of Client A -- Client A gets the entity information of Client B
• since both entities have ID 0 they clash on the other client's world

If my assumption is correct then you need a way to add arbitrary entities to your clients' worlds. Would a simple component be sufficient? Something like:

data class NetworkComponent(
  val clientId: String,
  val entityId: Int
)

The server will be able to assign a unique entityId and he also knows from which client it is coming. That way every entity gets marked with an additional unique identifier amongst all clients. Also, clients can still create their own "local entities" immediately. They get their unique NetworkComponent identifier on the next server sync.

The only thing you need is then an efficient way to retrieve an entity out of your world via NetworkComponent but this should hopefully also not be a big issue.

Let me know if I understood it correctly and if this would be a good solution for your case.

[Quillraven]

@Gidroshvandel How do you make sure that on each end only new entities are created which do not use an entity ID which might be in use on the other side? If you have game logic running on each side than both will create new entities on their own and try to sync it over the net with the other side. That will lead to a ID clash very fast.

Yes, this presents a problem, and I don't have a known solution yet... I will probably have to create an additional entity identification system that will be managed by the server Maybe something like artemis-odb solution junkdog/artemis-odb#267

@Quillraven I just had the idea that we could reserve a bunch of IDs for a world. So let's assume we create two worlds locally e.g. in a test program. Each world can create new entities with IDs from a list which is unique i.e. the IDs are exclusive for that world. Then it would be save if any world creates new entities that those entities when synced to the other world are not clashing. Thus we would need the feature to specify the range of IDs for a world at creation which this world is allowed to create. Other IDs can only get into that world from outside. Then we just need to make sure that both worlds use different ranges for their IDs.

When syncing entities from outside with other IDs then we need a function to specify the ID for the to be created entity.

I hope you got the point and that this works at all :)

This would work imo as well but I also see the problem about what should happen, when one world exceeds its entity limit and will then intersect an entity range of another world. We could say that's the programmers problem then and he needs to reserve enough space ;) But I think we can come up with something more robust.

By the way, thanks for joining in the discussioon :)

[Gidroshvandel]

https://github.com/Quillraven/Fleks/issues/80#issuecomment-1399326969

Not exactly, there will also be a world on the server, but without UI systems. The server also runs a simulation of the world to check that none of the clients are cheating. During normal operation of clients, the server will simply send the same data that is already predicted by the client and the user will not notice anything. But if he tries to cheat, he will not be able to have a detrimental effect on the worlds of other players.

Client A has its own local world Client B has its own local world Server has its own local world Client A creates an entity with ID 0 Client B creates an entity with ID 0 Server get event from Client A and create with ID 0 Server get event from Client B and create with ID 1 The server now synchronizes both of these worlds and sends updates: -- Client B gets the entity information of Client A -- Client A gets the entity information of Client B since both entities have ID 0 they clash on the other client's world and desync with server

But for operations between entities of different worlds, it is enough for us to know the id on the server, because it is a synchronization point, i.e. we can return an updated component to each client that will contain the id of the server entity and local client id, then the client will use it to communicate with server. And server will understand with which entities changes occur and there will be no conflict

the data structure will be something like this

Component in server world:

data class ServerNetworkComponent(
  val map: Map<String, Int> // where string client id key, int client entity id 
)

Component in client worlds:

data class ClientNetworkComponent(
  val serverEntityId: Int
)

And yes, thank you all for the discussion :)

[Quillraven]

Okay, so this component approach is already working for you? So you'd only need the method mentioned above to easily set components on an existing/newly created entity?

[Gidroshvandel]

While only in theory, I'm just working on its implementation. But in any case, I need a way to conveniently update entities in the world...)

I will write about the results and attach a link to the repository with the implementation, if it all works out)

[Quillraven]

will try to release a new version with the changes of the PR this weekend