Manual helpers for vector auto-inbetweening

[scorpion451]

I like to ink and color in a very freeform way, so most attempts to use the autotweening functions so far have ended up looking like tangles of spaghetti. I have a feeling if I could just manually give the system some reference points, it and I would get along much better. I'm picturing something like the tracking system or hook system, just some marks that tell the system "this point/area moves from here in frame 1 to here in frame 4"

[blurymind]

You need to get to know the tweening system before you can use it. The reason it turns to spaghetti for you is that there are things to watch for.

every time you make a line - that line has two variables that OT uses to tween:

1. The order number
2. The direction in which the line is drawn

So if you want a working tween - frame A and frame B would need to have the same order of line drawing and the lines the same direction of drawing.

Thats also how Cacani works too - but there its made much more intuitive and the tween feature is much more mature.

[scorpion451]

So if you want a working tween - frame A and frame B would need to have the same order of line drawing and the lines the same direction of drawing.

Yes. This is the core of the problem.

That is not a viable way of tracking linework drawn by someone who works organically.

I am not a machine- the order lines get drawn in, and the direction in which I draw them, are dictated by what I am drawing, and what the line needs to look like. These things are going to vary from frame to frame- it is a simple fact. Even in cases where I copy and paste a frame, I erase and redraw parts that change (it's exponentially faster and better looking than nudging a vector toward a rough approximation of where I want it.)

The best way to go about it would probably be to ditch the idea of duplicating the exact line segments entirely- just create a fresh set of shapes like it would create from a raster image, and then use that for the tweening. They aren't going to be the same in both frames anyway, so who cares if they get merged. I can always do manual cleanup if it merges them poorly, so long as it gets the majority of the deformations right. Better that than having to manually draw every tween frame.

[walkerka]

Recently I'm researching on this subject for my anime editor. It's very hard to auto match the strokes between frames. There's no reliable way to find the stroke pairs to generate inbetweens, even with user hint points. The direction can be auto determined by comparing the distance between end points though.

My solution is adding a "stroke" editing mode. It highlights current work stroke (also show corresponding stroke of nearby frames in onions). User can select one stroke and repaint the whole stroke (it still need to be one stroke).

It doesn't remove restrict 1) but at least users know where they are and don't need to duplicate drawings.

[blurymind]

@walkerka that is actually how Cacani does - The onion skin displays only 1 previous keyframe frame. It also highlights the one stroke from the previous frame (depending on how many strokes you have made so far) to the one that corresponds to the one that you are going to make. https://www.youtube.com/watch?v=NQEgL8tZg-0

[blurymind]

Edit - since my proposal is for a different feature to the one posted here originally - I moved it to its own place here: https://github.com/opentoonz/opentoonz/issues/565

To avoid confusion

[scorpion451]

None of this actually addresses the problem though. The number of lines is going to vary between keyframes, especially in the situations where autotweening would be most useful. I do not care how many lines or shapes it uses to imitate the shapes I draw. I just want it to make the stuff around point A move to point B over the course of z frames. To demonstrate, 1, 3, and 5 are keyframes, 2 and 4 are linear tweens.

So just throw out any reliance on line number/order/direction; it's not a useful value for determining what moves where. I, however, could easily point out to the program where corners and equivalent surface/outline points are, and give it a set of point-to-point rails to follow. The rest is a simple matter of some Lagrangian particle tracking over a point to point vector field- just basic vector calculus any graphing calculator can do. This would be a useful method of interpolation that doesn't require me to draw in an awkward and mechanical way that hinders my ability to produce quality work.

[blurymind]

@scorpion451 while our suggestion does not completely free you from drawing them in specific order - it makes it significantly easier to draw them in the correct order. It has also been proven to work on a large productions by other professional software - such as cacani.

I am not exactly sure how what you propose will work. If you find it difficult to explain- it is perhaps difficult to use and even more difficult to implement.

In any case I would take the opportunity to point the developers of opentoonz to a new vector technology that solely focuses on tweening. It's called vpaint - its still under heavy development - but it shows a lot of promise: https://github.com/dalboris/vpaint

The license is compatible - but it is so feature incomplete and unstable that I would advice against implementing it in OT at this point.

But to be honest - opentoonz already has the required technology to do this quite well without any external software - it just needs an onion skin mode that would aid it ( #565 ).

[scorpion451]

People used to store food in their basement with big chunks of ice, and that sort of worked for a long time too. Doesn't make it an efficient, convinient, or effective method.

The point system would be a fairly simple thing to program- if I had the time to burn on familiarizing myself with the program's code I'd do it myself.

Step one- generate a temporary mesh similar to the plastic tool using the guide points as the base vertices and subdividing. Step two- interpolate this grid from frame A to frame B with an appropriately weighted bias for each tween frame (factoring in the choice of linear, ease in, ease out, etc) Step three- apply the resulting vector field as a coordinate- based matrix to the vertices of whichever frame has more line segments, just like you would with a warp tool in a 3d program. (Allow manual overrides of this option for the inevitable times when the one with more lines is the wrong choice) Completely ignore what the vertices are part of- only their position on the vector map is relevant. Just look at where they are, move them to the equivalent point on the distorted grid, and smooth them a little if they aren't right next to a landmark. Step four- print the result to the tween frames. Step five (optional)- run a light "smooth" algorythm to buff out any weird crumpling that may have happened along the way.

For outside inspiration, Gimp's animation package has had a system almost exactly like what I'm suggesting since, like, 2006 or something. Makes the tweening process a breeze- make your keyframes, put some marks showing where the important landmarks are, hit button, get tweens that need only a little cleanup. This would just be adapting the same thing to vector, meaning no pixilation from heavy deformation.

Gimp's cage tool would also be a good thing to look at, as it uses some even fancier deformation algorithms to do something similar, but with a basic understanding of perspective built in.

[blurymind]

@scorpion451 I dont mind drawing the lines in a correct order myself :)

Its just currently missing the ability to show the order like some other software - Thus why I suggested it. it will compliment the current system in a way that improves it 100%.

I demonstrated with a video how it works in cacani, with an animated gif - in tween maker. Both of these applications can be downloaded and tested.

I am not saying mine is better - it would be good to explore both ideas. But I still have trouble understanding how yours does it's magic. Gimp is an opensource example - which is great! It's a good reference. But I dont remember seeing any animation tweening capability in it. Can you link us to a demonstration video of how gimp's tweening animation system works?

[walkerka]

@scorpion451 I got your idea. It looks more complicated to implement. (at least for me) It need build segments mapping automatically between frames by mesh grid.

The hinting rule is not so intuitive. Users may give invalid input (By using overlapped or twisted hinting points). Users also have to mark enough hint points to make it work. And these hint points must follow the same order. But how can we know it's "enough"? By drawing and playing inbetween again & again. In worst case, it may be even slower than my approach.

I'm not saying the stroke order based approach is better. It only solves very limited inbetween generation (without large turns/perspective changes). IMO auto inbetween function should only serve as a utility for inbetweener, not a major work flow. Users should be able to combine auto inbetween with their hand crafted drawings easily. We should not force user to use auto inbetween for the whole drawing. It need more planning before drawing. It should only be applied to the most stupid and tedious parts.

@blurymind My solution is not fully implemented yet. I'll explain more in #565

[ghost]

Based on the discussion in #1207 feature requests are going to be closed here on GitHub unless there is a developer actively working on the feature. Feature requests can be discussed at the OpenToonz Google Group: https://groups.google.com/forum/#!forum/opentoonz_en