Animation Synchronisation

[tospie]

Server should provide a mechanic to synchronize playback of (Xflow-)animations between clients.

Using this animation plugins, other plugins as well as clients should be able to trigger animations of entities to start or stop. Server should then synchronize the animation state back to any connected client.

[rryk]

How do you plan to synchronize the animation state? Will there be a fixed parameter (such as time) or a generic approach to synchronize a set of arbitrary parameters?

[tospie]

To the current state, the plugin is designed to synchronize Xflow-animations, thus no rigid body animations that are realized by just changing position or orientation parameters.

Currently, there is a keyframe parameter that specifies which keyframe of an animation is currently to be displayed for an entity. This keyframe is then sent to the clients on every update. This approach is currently implemented.

Another approach, which may also be included in the plugin, is to just send start- and stop commands for animation to the clients. The clients do then individually compute the keyframes locally.

First approach is better for situations where it is important to really have same keyframes on each client, like for example exact collaborative simulations where each collaborator needs to see the exact same state.

Second approach is fine for animations that are less crucial, for example a walking or waving avatar. Putting keyframe update computation on the client reduces computational workload on the server and number of update messages (as for server side animations, keyframe updates need to be sent with at least 25 fps for real time rendering)

[rryk]

First approach is also useful when the length of the animation may be variable (such as walking for different movement speeds). What about the case when there is more than just one parameter, e.g. sync all bone positions in a skeletal animation?

[tospie]

Currently all this bone stuff is expressed via Xflow operators in the XML3D model, and the current plugin aims at realizing animation with that, i.e. for the moment, there is no plan to extend the current animation plugin to support server side synchronisation of skeletal animation.

I think it would therefore make sense to rename the plugin to something like KeyframeAnimation or XflowAnimation (as the plugin has to include some concepts of how to define Xflow animations to make it work with the client).

Maybe we could even split server-side keyframe synchronisation and client-side animation invocation into two different plugins. Whereas the one triggering the client-side animations does not even need to limit the commands it sends to the client to animation-related commands.

For the cases of more parameters, I definitely see a plugin of its own, like SkeletalAnimation, RagDoll or something. One could also think about a generic plugin that introduces bones to entities, and plugins that build on that to operate on the skeletons.

[rryk]

I see your point. Let's then just keep it simple for now, but I see the need for more advanced animations later. However, I didn't mean that a server will drive the animation. It will simply synchronize a set of parameters as dictated by the client or some other plugin, e.g. Avatar.

[tospie]

Oh, ok. So just a plugin that provides components to contain the bone parameters. Nice idea, client side computations keeps the load on the server low then.

[rryk]

I actually haven't though much about Animation plugin design yet. After looking at your code, I am not sure if this is the right design. It limits number of the animations per object to one and doesn't allow to have advance animations, such as ones that depend on more than one parameter. Also I am not sure if animations need to be driven on the server. Why not on the clients? Btw, same applies to the Motion plugin.

I will think more about what a good design would be in a few days. Today I need to focus on writing.

Sergiy

On Wed, Jan 8, 2014 at 10:24 AM, tospie notifications@github.com wrote:

Oh, ok. So just a plugin that provides components to contain the bone parameters. Nice idea, client side computations keeps the load on the server low then.

— Reply to this email directly or view it on GitHubhttps://github.com/rryk/FiVES/issues/57#issuecomment-31812942 .

[tospie]

The current design is tailored very much towards Xflow-Keyframe-Animations, as this is the kind of animations we will have to deal with in our Web client. Those allow only one Xflow-key to steer the animations, therefore there is technically only one animation. You can however encode several animations in a key sequence, e.g. "walk" is realized by key frames 0 to 2.1, "wave" from 2.1 to 2.7 and so on. This partitioning is defined in the XML3D model file and read by the client which then invokes the respective animation on the server with the correct key range.

Advanced animations based on several parameters are also not tackled in Xflow yet, to my knowledge. At least the current operator for Skinning is driven by key frame only.

That's why I suggested to rename the plugin to XflowAnimation.

As stated in one of my previous comments, both client- and server-side driven animations will be provided by the plugin. There will be the option to have the animation entirely driven on the server, to have it driven on the client with synchronizing the key frames to other clients, and the possibility to have the server just start- and stop messages and each client computes their key frames independently.

Reasons for introducing the server-side computation are also given above: Existing use cases already demand for a technique that computes the frames on the server to ensure that every client sees exactly the same animation state, e.g. for collaborative inspection of some simulation.

[rryk]

Hi,

The current design is tailored very much towards Xflow-Keyframe-Animations, as this is the kind of animations we will have to deal with in our Web client. Those allow only one Xflow-key to steer the animations, therefore there is technically only one animation. You can however encode several animations in a key sequence, e.g. "walk" is realized by key frames 0 to 2.1, "wave" from 2.1 to 2.7 and so on. This partitioning is defined in the XML3D model file and read by the client which then invokes the respective animation on the server with the correct key range.

What if an object has several animations that can be played at the same time? Say walking and waving? This can easily by achieved by morphing.

Advanced animations based on several parameters are also not tackled in Xflow yet, to my knowledge. At least the current operator for Skinning is driven by key frame only.

That's why I suggested to rename the plugin to XflowAnimation.

Xflow is not limited to single-parameter animations. It can be used with multiple parameters as well. Therefore it would be XflowSingleParameterAnimation plugin. However, I am considering designing a universal plugin that can support any number of parameters per object.

As stated in one of my previous comments, both client- and server-side driven animations will be provided by the plugin. There will be the option to have the animation entirely driven on the server, to have it driven on the client with synchronizing the key frames to other clients, and the possibility to have the server just start- and stop messages and each client computes their key frames independently.

Reasons for introducing the server-side computation are also given above: Existing use cases already demand for a technique that computes the frames on the server to ensure that every client sees exactly the same animation state, e.g. for collaborative inspection of some simulation.

Why not start simulation on all clients at the same time with a start message? Optionally the speed, exact start time and number of repeat times can be transmitted together with this message. This saves a lot of bandwidth as only one message needs to transferred instead of multiple tests. I have used similar approach in my

paperhttps://graphics.cg.uni-saarland.de/fileadmin/cguds/papers/2011/byelozyorov_cw2011/4467a046.pdf

see animation configuration dialog on Figure 4 (left).

— Reply to this email directly or view it on GitHubhttps://github.com/rryk/FiVES/issues/57#issuecomment-31822073 .

[tospie]

I see your points, but note that animation plugin is not finished yet, that's why it is still an open issue and no pull request :)

Why not start simulation on all clients at the same time with a start message? Optionally the speed, exact start time and number of repeat times can be transmitted together with this message

See my posts before, this feature will also be included in the plugin. It is just not implemented yet.

Also I am not sure if animations need to be driven on the server. Why not on the clients? Btw, same applies to the Motion plugin.

Also, see my posts and our discussions before. Of course they don't need to be driven on the server in general, but in some of the use cases I have. The finished plugin should cover both server- and client-driven animations.

[tospie]

Line breaks fixed, branch tested, merged and closed.