Client-side physics sim screws up streaming

Mavor commented 12 years ago

If a client-side physics sim is started up, it completely wrecks the streaming. I believe this is because entities are streamed in as-is (just their current translation data) and nothing is sent into the client-side box2d. Thus, the box2d physics thinks this new entity is always at 0,0 (because no box2d translation data is performed).

Rob, I know that you are trying to basically have the client simply be a receiver for all data (and do little-no processing by itself), but for more intensive games, we might want to offload some of the physics processing work to the client... What we really need is an option on the stream component to tell it to directly modify the box2d body positional data (somehow incorporate the currently included interpolation data).

If we don't have an option to write stream positional changes to the client's box2d body instead of it's entity transform, then we will need to set up custom data on the stream and then manually modify the box2dBody position ourselves, correct?

What do you think?

cratica commented 12 years ago

Except for ultra simple games, you definately want local physics processing. That can really chew up the bandwidth in a hurry. No reason you can't process it on the local machine and update basic position/rotation information. Perhaps have options for what data to send and how often to send it. Some games won't need to update positional data 60x second so for optimization purposes, there should be an option on how often to relay the physics generated updates back to the server. Just my 2 cents (worth about a penny)

Mavor commented 12 years ago

Yes cratica.. physics processing is costly, and instant feedback is needed. Isoengine doesn't support pushing a client stream back to the server yet (if ever) so i created my own "stream". Unfortunatly this means I need to code in my own interpolation :*(.

Perhaps later if Rob feels this is a worthwhile addition to the engine we can just create a stream that goes from client-> server instead of server->client using the existing infrastructure somehow. :)

Irrelon commented 12 years ago

@Mavor Box2d not being updated is a bug. Once the interpolation system is in place it will be fixed. Client-side box2d entities will receive the same transform updates as the parent IGE entity.

You shouldn't send position data from the client to the server you should send client game control input data using standard network commands (which are just as fast as using the stream).

Back to updating the position of box2d entities... keep in mind that box2d doesn't "want" you to simply set the position of a body... it doesn't expect that body's will update their position from an external source... in fact box2d expects to be authoritative. This means that if you simply set the position of a body box2d will not process physics for the movement from its previous position to the new one and that can mess with the overall physics sim... this is just the way box2d is. There are sort-of ways around it... including working out the vector from current to new position and applying a physics impulse to move that distance.

Irrelon commented 12 years ago

@cratica Can you give me an example of where you need both server and client physics sims running at the same time? Other than if you're employing client-side prediction I cannot think of a reason myself.

Also you might have misunderstood the stream system, it doesn't send updates 60x a second unless you tell it to... the default is 20 times a second but you can set it to less or more depending on what your game needs. There is a section of the docs related to the stream here: http://www.isogenicengine.com/documentation/isogenic-game-engine/versions/1-1-0/manual/networking-multiplayer/realtime-network-streaming/ - the stream is not tied to the renderer in any way and runs it's own separate timer.

cratica commented 12 years ago

No, not simultaneously. Great!! on the ability to set streaming needs. My point was that server sim IS the better way to go if bandwidth isn’t an issue and if running only a few instances. For complex games, it needs to be client processed for the reasons mentioned.

Thanks a bunch!

From: Rob Evans Sent: Wednesday, September 12, 2012 9:18 AM To: coolbloke1324/ige Cc: cratica Subject: Re: [ige] Client-side physics sim screws up streaming (#34)

@cratica Can you give me an example of where you need both server and client physics sims running at the same time? Other than if you're employing client-side prediction I cannot think of a reason myself.

Also you might have misunderstood the stream system, it doesn't send updates 60x a second unless you tell it to... the default is 20 times a second but you can set it to less or more depending on what your game needs. There is a section of the docs related to the stream here: http://www.isogenicengine.com/documentation/isogenic-game-engine/versions/1-1-0/manual/networking-multiplayer/realtime-network-streaming/ - the stream is not tied to the renderer in any way and runs it's own separate timer.

— Reply to this email directly or view it on GitHub.

Irrelon commented 12 years ago

@cratica I thought you meant client and server physics at the same time! :)

Thing is... unless I misunderstand you again (which is totally possible), running the physics on the client and then sending position updates to the server is exactly the same as doing it the other way around isn't it? The exact same bandwidth is used regardless of which way you do it....

cratica commented 12 years ago

Depends on how complex the physics requirements are. If you are playing a game that requires physics calculations on the client side that aren’t required on the server side, then it’s “cheaper” to run physics on the client side. Also, if it’s a complex game, performing full physics calculations on 100 characters for example on a server, then spitting that out to each client would be very taxing on the server and bandwidth. However, if each client calculated its own physics, less bandwidth, less server overhead. An example would be 100 characters with full bones with full physics on everything (extreme example for argument sake). However, it could be that the only data needed for sharing are pelvis (location), arms and legs.

It really all depends on the application and optimization. I think for most (if not all) Facebook games, server side is the best solution since these games are far simpler. So it depends on what you see the future base for the engine to be.

Sorry if I’m making it more confusing. I’ll shut up now...

From: Rob Evans Sent: Wednesday, September 12, 2012 11:53 AM To: coolbloke1324/ige Cc: cratica Subject: Re: [ige] Client-side physics sim screws up streaming (#34)

@cratica I thought you meant client and server physics at the same time! :)

Thing is... unless I misunderstand you again (which is totally possible), running the physics on the client and then sending position updates to the server is exactly the same as doing it the other way around isn't it? The exact same bandwidth is used regardless of which way you do it....

— Reply to this email directly or view it on GitHub.

Irrelon commented 12 years ago

@Mavor No no, it's good to have a discussion about all of this, every opinion matters! :)

Don't think any game sends every bone or joint update to the client. There are pre-defined actions like "pick up object" or "run" or "walk" that are animations fired when the server sends the state change for the player.

I've tested 100 characters in a box2d sim server-side to 100 clients... it's not taxing. Even on a mini instance on the rackspace cloud the server only used about 60% cpu and bandwidth wasn't an issue. A standard box has 100mbps upstream (what the server can "send"). Each entity update contains about 20 bytes for an average when moving and rotating. Assuming 20 updates a second that's 400 bytes a second per entity. Assume 100 entities that's 40kb per second per client so with 100 clients that is = 4mb a second. Upstream of 100mbps can push around 12mb a second leaving lots of megabytes left to play with.

That said, you're never going to create a game where all 100 players are in the same part of the game as the rest of the 100 players at the same time from a single mini-cloud instance because that would be silly. Making sure you set some decent interaction radius values you can limit which entity updates are sent to which clients usually reducing this traffic to about 1mb / 2mb a second.

In other words... it's doable even off a mini cloud instance!

cratica commented 12 years ago

Thank you Rob. I’ll be the first to admit that my opinion on this should not count for much since I am coming from this as a game designer/artist rather than a programmer. (although I have programmed a few games). I do think this is important though and I want to help make your engine as flexible as possible.

There are definitely pros/cons to both methods. I brought this up with a friend of mine that I’ve worked with at 2 different game companies through the years in California and Texas to see what his opinion was. He’s one of the brightest game programmers I know and has worked on some top titles including Medal of Honor while working at EA.

I basically just told him I knew someone working on a Facebook engine and we were discussing pros/cons of client/server physics and that I thought client side physics would be more powerful and less bandwidth, just sending out location and other critical info versus having the server side do everything. (btw, I totally agree that in a perfect world Server side would be ideal and I believe for the purposes of Facebook, server side is the right call but I think it should be left open to support client side as well. That would give any future developers the choice depending on their application ). Anyhow, his response is below.

“Yeah, what you said is right, client side physics you can do a lot more and probably reduce bandwidth (doing anything more than really simple physics on the server is going to be expensive - but I don't know how you're thinking of scaling on the server side).

BUT, if the physics are important to the outcome of the game and you don't want people to be able to cheat, then you have to do it on the server side. Doing it on the server also makes the client more complicated because presumably you want them to be smooth so they have to predict the physics outcomes and then resolve in some nice way the differences between what the server computed and what the user is seeing.”

Steve

From: Rob Evans Sent: Wednesday, September 12, 2012 12:21 PM To: coolbloke1324/ige Cc: cratica Subject: Re: [ige] Client-side physics sim screws up streaming (#34)

@Mavor No no, it's good to have a discussion about all of this, every opinion matter! :)

Don't think any game sends every bone or joint update to the client. There are pre-defined actions like "pick up object" or "run" or "walk" that are animations fired when the server sends the state change for the player.

I've tested 100 characters in a box2d sim server-side to 100 clients... it's not taxing. Even on a mini instance on the rackspace cloud the server only used about 60% cpu and bandwidth wasn't an issue. A standard box has 100mbps upstream (what the server can "send"). Each entity update contains about 20 bytes for an average when moving and rotating. Assuming 20 updates a second that's 400 bytes a second per entity. Assume 100 entities that's 40kb per second per client which = 4mb a second. Upstream of 100mbps can push around 12mb a second leaving lots of megabytes left to play with.

That said, you're never going to create a game where all 100 players are in the same part of the game as the rest of the 100 players at the same time from a single mini-cloud instance because that would be silly. Making sure you set some decent interaction radius values you can limit which entity updates are sent to which clients usually reducing this traffic to about 1mb / 2mb a second.

In other words... it's doable even off a mini cloud instance!

— Reply to this email directly or view it on GitHub.

Mavor commented 12 years ago

As Cratica said, there are some things (especially processor heavy things) that can be offloaded to the client.

I just wanted to pipe in that even if a player is theoretically able to hack something, there are ways to counter his hacking. For instance, in Soccer, if a player breaks the rules, they get a foul. I don't see why the same principle can't be used in a game, and we check the mechanics that the player's client has some control over periodically to watch for abnormal behavior.

When you get to making complex, high-user-count multiplayer games, the bandwidth/processing saved on a per-user basis really starts to add up, and at that point, the coding time spent putting together anti-hack/"referee"-like systems on the server is much less costly then buying a ton of servers due to processing all being done on the server.

Finally, as Cratica said, there are some cases where you want the client to see a very elaborate, complex physics sim/animation/effect/etc... without having to calculate everything on the server. In these cases (and especially for graphical/local flavor-only systems), it makes sense to allow the client to handle it's own thing.

Irrelon commented 12 years ago

So basically I see a few scenarios...

You are writing an MMO, in which case you are going to section off your world into small maps (lets say 200 x 200 tiles - 40,000 tiles total) and players will play together in teams so you can create world instances for each group of players (probably not more than 8 - 16 players at a time). Each group is basically playing the same "map" but they are only playing multiplayer with their group.

In the other instance you are also creating an MMO but you have a large well-visited town and a lot of players are likely to visit this town at the same time. In that case you offload processing of the town to different servers. Only 100 players at once are ever on the same "town" server. If you are grouped with friends then they are all on the same server.

If you are not creating an MMO but rather just trying to sync 16 players together (like for instance medal of honour) but using javascript you still wont have client-side physics processing. The reason is this...

On medal of honour (native game written in C++) you have a closed-code environment where hacking the code is probably not a viable option for 99% of players. The network data is something you can hack but there are checks in place to ensure you don't teleport such as checking previous position against new position vs time.

The native games will have client-side physics as well as server-side physics to ensure that client-prediction works but the client will still never send position data back to the server, or if it does it will be THOROUGHLY checked by the server before being accepted.

What I'm trying to get at is that unless there is some fundamental need for it, client-side to server-side streams are kinda pointless. If you REALLY want to do it I can add it to the engine but I really don't see the need. Based on my calculations a mini-cloud-hosted instance should be able to support around 1,000 players on average without issue, costing around $12 a month to host. If only 5% of the players are monetisable and spend 1$ each a month that's still $50 a month so you're still covering running costs and making a profit. At any given time you might have 30% of those users accessing the server but that is still well within the bandwidth tollerance of the server and if it's not, just fire up a new instance!

Remember that 1,000 players are unlikely to all play the game at the EXACT same time so you'll have an organic distribution of player usage.

If you REALLY REALLY think that client to server streams are important, I'll add it to support you but you might be opening a Pandora's box by allowing that kind of data flow. :)

cratica commented 12 years ago

I don’t need it personally, I was just pointing out that it might be important for the growth and flexibility of the engine because there will be those who feel they need it. Just trying to help.

From: Rob Evans Sent: Wednesday, September 12, 2012 8:41 PM To: coolbloke1324/ige Cc: cratica Subject: Re: [ige] Client-side physics sim screws up streaming (#34)