Spine support for duality

[HeadClot]

Hey Adam -

I am making this request for a tool called Spine which you can find here - Link

It is focused on 2D Animation for games and making it easier on artists. You can find the various runtimes here as well.

If some sort of plugin could be made for Duality that would be awesome.

• HeadClot

[AdamsLair]

Hi HeadClot,

I know Spine and think it is an incredible toolkit for 2D games. Would love to support it in Duality! :)

It's not a high-priority issue right now, but at the same time it shouldn't be too hard to implement a third-party plugin adding Spine support to Duality. As far as I know, @BraveSirAndrew and @Andrea have already used Spine (or something similar?) in their Duality game Onikira. Maybe they can share some experiences on this? :)

Although I probably won't get around to it anytime soon, this would be an excellent opportunity for contributions via self-contained third party plugin. And of course there'll be support in the forum for anyone getting into this!

[HeadClot]

There is also Spriter - You can find that here - http://www.brashmonkey.com/ It exports various 2D Animations as .PNG but it also has its own API.

You can swap things in real time between the tool and game engine. It is also cheaper while in early access.

Its API is publicly available (I think) and allot cheaper than Spine. Only 25 bucks.

Here is a video about what I mean about the runtime features. https://www.youtube.com/watch?v=45na21YFTwg

[AdamsLair]

Thanks for bringing this up! I've seen Spriter a while ago and it looks like a great toolkit as well, although I personally have a better feeling about Spine. Since both Spriter and Spine integration would take place in a distinct plugin, there is no reason not to have both. :) Just need someone to implement it.

Asset hotswapping should be no problem in any case. If Duality detects a source file being modified, it reloads the associated Resource anyway, so this should be more or less a "free" feature. The only problem here might occur if Spriter or Spine store their actual animation data in multiple files, because the current Asset Management system in Duality is pretty trivial and doesn't expect a Resource to have multiple source files. Should be fine for sprite images though, because they'll be distinct Resources anyway.

[BraveSirAndrew]

Hi @HeadClot

@AdamsLair is right in that we do use Spine in Onikira. Our implementation is unfortunately more tightly integrated to the rest of our game and our fork of Duality than I'd like right now (not my best code ever! :)) so even though pulling it out into a separate plugin is on my todo list, it's not likely to happen any time soon:( We could totally help out by answering questions and posting snippets of code though, if that would be of any use to you?

Spriter looks pretty cool and certainly seems to have come a long way since the last time I looked at it.

[AdamsLair]

We could totally help out by answering questions and posting snippets of code though, if that would be of any use to you?

Merely thinking about how such distinct plugin could - in theory - be implemented, what is your approach on skeleton representation in Duality? There are two ways that come to my mind:

1. Have a self-contained SpineAnimation Component that does all the rendering and updates and acts mostly as a blackbox. Content details like bones are hidden from the outside for higher flexibility, although queries like "Where is position X relative to bone Y right now?" are possible via code.
2. Have a SpriteAnimation Component in a "main" GameObject that actually unwraps the whole skeleton structure as a set of automatically generated child GameObjects that are modified during runtime according to the animation. Full access is granted, including supplying sub-objects with custom Components or shooting yourself in the foot heavily.

My personal favorite would be version one, but I acknowledge that the second one may also have its advantages - provided, you can handle the disadvantages in a graceful way. What's your take on this and how did it turn out? What would you do differently if you started over?

[BraveSirAndrew]

The first big mistake we made was trying to make a unified frame based sprite and spine animation system. When Onikira was just a baby, it used purely frame based sprite animation, because of time, money, and talent constraints. When we switched to Spine, we tried to extract some interfaces from our sprite animation stuff, including making a base resource class to represent abstract animation resources, and implement them in Spine. That was a mistake! :) Keeping those separate would have made way more sense.

We actually ended up with the first approach you mentioned above. We have a SpineRenderer component (which actually does all of the animation as well as the rendering, so maybe the name isn't great but SpineAnimation would also do rendering so you can't win:) ). Bone info is not exposed from this class, but we have some other components (SpineBoneTracker, SpineTrailEmitter, and SpineCollider) which can be used to attach either special case components (trail emitters and rigid bodies) or any other game object to specific bones. These components come with their own property editors that let us select what bone to attach to from a drop down box. This all works reasonably well. The rendering code is fairly interesting, especially after we added support for free form mesh deformation and skinning. Spine exports vertex data ready for indexed rendering, but because Duality doesn't support indexed rendering yet (that I can tell. that right @AdamsLair ?) we do some creative tinkering to create the right verts every frame.

There are some cases where it would be better to have gone with the second approach though. For some of our giant boss characters, we can end up with depth ordering problems. For example, one character walks along in the background and tries to side swipe you off buildings. As it swings its hand out, we'd like it to appear in front of the building that you're standing on, but because it's one single game object, we can't do that and keep the torso behind the building at the same time. Additionally, we wouldn't need special purpose bone attachment components with the game object hierarchy approach. You can just attach things directly to game objects and Duality's transform code will take care of it. And there's more - detachable body parts, applying different materials to different bones, etc. I've spiked some of that code and it doesn't seem that it would be all that difficult to do, but some sticking points are how to deal with bones that are new/deleted/moved-within-the-spine-hierarchy on re-import, and ...actually that's the only one I can remember right now:)

[AdamsLair]

Wow. This is a great piece of insight. Thanks! :)

Duality doesn't support indexed rendering yet (that I can tell. that right @AdamsLair ?)

Correct. It was never really required yet, so there's no support. Doesn't have to stay like this though - we should get talking on how to improve this once we get around to it :)

it doesn't seem that it would be all that difficult to do, but some sticking points are how to deal with bones that are new/deleted/moved-within-the-spine-hierarchy on re-import

Indeed. Deleting bone GameObjects that the Spine base Component still needs, or moving them, or making a Prefab out of a Spine object and then changing it, or attaching more Components that change their parents at runtime, etc... it would be a nightmare to make sure it's always synchronized without crashing.

Not only because of those concrete use cases, but also because of Dualitys workflow in general: Everything you've learned in that engine aims at having passive Components that never do stuff on their own and performing active work in the editor. Introducing a different paradigm for a single Component is a solid breach in user experience. I'd rather avoid that for the better of the framework as a whole.

One way to get around this is to have a blackbox Component that exposes an interface to optionally specify child objects for bone attachment points - which would allow the user to mirror the relevant parts of the skeleton in the editor without having an "active" Component or forcing perfect sync.

But then again, that BoneTracker of yours is probably the same thing in a different shape. All in all, your approach doesn't sound too bad :) Maybe this is a solid starting point: Having a blackbox SpineRenderer Component and allowing special functionality through additional Components.

One could even think about splitting it up even further and having a SpineSkeleton Component and a SpineRenderer Component, where the first just updates the animation and skeleton and the second one does the rendering. In certain cases, it could be left out and be replaced or extended by a SpineTranslator (or something) that instead synchronizes some explicitly stated GameObjects? It still sounds quite similar to your approach though.

As it swings its hand out, we'd like it to appear in front of the building that you're standing on, but because it's one single game object, we can't do that and keep the torso behind the building at the same time.

That should be no big deal, actually. A single ICmpRenderer can add any number of DrawBatches in its Draw method - and since Z ordering takes place on a per-batch level and not a per-object level, you can have your blackbox render at pretty much any Z coordinate with correct sorting. You just need to split its drawcalls for each "layer".

Alternatively, if you're using a DrawTechnique that doesn't require Z-Sorting (like Mask) you will be able to rely entirely on the Z-Buffer and thus have correct sorting even with a single DrawBatch. I suppose you're using Alpha techniques though, because otherwise you probably wouldn't have had that problem in the first place.

Still, the split method should do the trick :)

especially after we added support for free form mesh deformation and skinning

Since Spine provides vertex data directly, what part of the Duality integration of Spine had to change in order to support this? I would have expected this to be already covered simply by using the provided vertex data..?

[HeadClot]

@BraveSirAndrew - I am really looking forward to your game. I just saw it on Steam early access.

[AdamsLair]

As seen here, @eriksk has started developing a third-party plugin for Spine integration in Duality that might solve this issue in the long run. Standing by for future updates.

@eriksk: Feel free to use this issue / thread for discussing your plugin as well. There might also be some real-world use case insights here that might be of interest.

[Andrea]

Hi there

We have this working ourselves, and we want to open source, it problem is it takes a little while, I can spend a few hours and start it off and maybe someone can take it from there? It is not pretty but it does work and might save a lot of time, What do you think?

EDIT: sorry, I just saw Eric has this working, nevermind, sorry we were so slow :( Andrea

On 2 February 2015 at 18:32, Adam notifications@github.com wrote:

As seen here http://forum.adamslair.net/viewtopic.php?f=18&t=362, @eriksk https://github.com/eriksk has started developing a third-party plugin for Spine integration in Duality that might solve this issue in the long run. Standing by for future updates.

@eriksk https://github.com/eriksk: Feel free to use this issue / thread for discussing your plugin as well.

— Reply to this email directly or view it on GitHub https://github.com/AdamsLair/duality/issues/121#issuecomment-72510711.

[AdamsLair]

EDIT: sorry, I just saw Eric has this working, nevermind, sorry we were so slow :(

Don't worry about this. You've got Onikira to take care of and you're probably loaded with work even without this.

We have this working ourselves, and we want to open source, it problem is it takes a little while, I can spend a few hours and start it off and maybe someone can take it from there? It is not pretty but it does work and might save a lot of time, What do you think?

I think open-sourcing it can't hurt in any case - if you can somehow manage to pull this off without doing more overtime.

As far as I know, @eriksk has done a first rough setup, but is still experimenting with the details - having another open source implementation around could be a helpful resource for refining them. I certainly can't speak for him, but I guess having it around and available would be nice..? :)

Worst that could happen is no-one gets around to do more Spine stuff, and in that case, you can always come back later and polish the already released source code.

[eriksk]

I haven't abandoned the spine project, just a lot of other stuff going on. Still need to refactor a lot to make it more user friendly. I also spoke to the spine people at esoteric and as I suspected, I cannot redistribute the spine binaries (I can, but I kind of don't want to..). There was a lot of ifs and buts in order to do so. But I think that since the spine runtime is on nuget, it would be simple enough to just add a how-to in the description for the plugin.

I'll get back here once there's more progress to report.

[AdamsLair]

But I think that since the spine runtime is on nuget, it would be simple enough to just add a how-to in the description for the plugin.

If it's on NuGet, the easiest solution would be to add a dependency to your own Spine plugin NuGet / Duality package - it will be downloaded automatically by the package manager and the end user won't have to do a thing. :)

[eriksk]

If it's on NuGet, the easiest solution would be to add a dependency to your own Spine plugin NuGet / Duality package - it will be downloaded automatically by the package manager and the end user won't have to do a thing. :)

That's the thing though, they won't let you use the binaries unless you have a spine license, it's a bit sketchy in how they explain it. But at least that's my understanding.

This is from the spine site.

The official runtimes are available on GitHub and licensing Spine grants permission to use the runtimes in your applications.

So I guess you are allowed to download but not use them :)

[AdamsLair]

That's the thing though, they won't let you use the binaries unless you have a spine license, it's a bit sketchy in how they explain it. But at least that's my understanding.

Hmm.. couldn't it be sufficient to include a license note in your plugin description that clearly states that the user will have to aware of the Spine license? Still, a small description text like this could go unnoticed... :/ Not a good way.

Here's the Spine License, as linked from their NuGet package. I'm really not sure how to read that. On one hand, it states that:

Without the written permission of Esoteric Software (typically granted by licensing Spine), you may not (a) modify, translate, adapt or otherwise create derivative works, improvements of the Software or develop new applications using the Software or (b) remove, delete, alter or obscure any trademarks or any copyright, trademark, patent or other intellectual property or proprietary rights notices on or in the Software, including any copy thereof. Redistributions in binary or source form must include this license and terms.

Which clearly wouldn't allow adding their binaries to your package without adding their license note and explicitly linking to them as well. But if you add a dependency to their package, you are not redistributing it. You are telling the NuGet installer to install their package for the end user, which should be fine for personal use:

You are granted a perpetual, non-exclusive, non-sublicensable and non-transferable license to use, install, execute and perform the Spine Runtimes Software (the "Software") and derivative works solely for personal or internal use.

The problem is though, because Duality doesn't pop up a license agreement dialog, the user won't ever see the Spine license unless actively looking for it. This may be a problem.

So, my question is: If I implemented a license popup for NuGet packages that require license agreements, would it be fine if you added a direct dependency from your plugin package to their NuGet package?

[AdamsLair]

Pinging the Esoteric Software guys, so they have the chance to weigh in here personally: @NathanSweet @magro

[AdamsLair]

I have updated Duality in order to display a license agreement dialog whenever a NuGet package states that it requires one. So when installing your Spine plugin, which references the Spine NuGet package as a dependency, the end user would get something like this:

If an end user doesn't accept the license, the package won't be installed. The update is already "live".

[eriksk]

That's perfect! That should be enough to include the nuget reference, nice work :)