Add transition effects

[derpeter]

This was suggested in the IRC by @CarlFK

It would be be nice to have transition effects when switching from one input / composite mode to another. This will need changes in the Core and the GUI.

[CarlFK]

There are 2 use cases for this:

1. hard cuts are jarring to the viewer. it is a surprise for the brain, eats up cognitive cycles. (or something)

2. I have often wanted to try blending the PnPs: fade a bit, then leave it maybe green screen floating head, maybe just giving the viewer a hint what is behind the PnP. Tthis would need a slider so I can fade in the image a bit, then leave it alone. which would add another UI control to the GUI, which I don't really want. I may find none of this is useful, but I would like to try.

[yoe]

(background: Carl asked me to comment on this. I hacked up the original fading transition for DVswitch, which then got repurposed into a "blend two video sources together" kind of thing. When I later ended up maintaining DVswitch, I regretted the way the whole thing was implemented. Having said all that, while I understand the general gist of how vocto works, I have never tried to set it up myself or looked at the code, and so don't have a detailed understanding of how it works. Please keep that in mind if I say something stupid below ;-) )

I think if you're going to do this, you should keep in mind that a "transition" and an "effect" are two separate things. Even if the backend might use the same video manipulation in the end (e.g., a fade transition would use linear interpolation between pixels of two video source frames, and so would a blend effect), the system should keep the two separate, because the failure to do so in DVswitch caused UI confusion, various bugs, and general unhappiness.

A transition is a way to go from one setup using one or more sources to another setup. Currently, voctomix supports only one type of transition, the "hard" cut (aka "no transition"). Possible other transitions are:

• Time-based crossfade
• When moving to or from pic-in-pic, a transition could gradually shrink the B source to its new location, revealing the A source which may or may not also shrink slightly (or vice versa)
• A swipe effect, whereby one source is moved out towards the left and the other is moved in from the right, with no scaling

etc. The common denominator here is that each transition is time-based, and a way to go from one to the other.

An effect, otoh, is a static setup for two video sources that get combined into a single output source. PIP is an effect; blending two sources together is another effect. While it munges video frames in somewhat the same way that transitions do, there is a difference in how the transition is controlled.

With DVswitch, I made the mistake of not recognizing that, and repurposing the existing effects framework for adding the crossfade transition. The result was that the crossfade could not be used for everything:

• Since an effect for DVswitch was defined as a result which took one or two input video frames and which produced an output frame, and since effects could not be stacked, that esssentially meant that crossfade transitions could only be requested from one source to the next, not from a PIP setup to another source. DVswitch tried to hide that by hiding the small pic in the PIP setup before transitioning to it. Since this was generally a bad idea anyway, most people didn't use it.
• The same was true for the "blended" effect; you could not crossfade from the blended effect to another situation. The effect would be that DVswitch would first switch to the non-blended version of one of the two sources (the "A" source, whichever it was, which might have been the one you would not expect), and then gradually fade to the other source.
• Since the crossfade transition reused the effects framework, it had to deal with the "A" and "B" sources. After the transition had finished, the UI had to be updated so that the current sources would reflect the situation as expected by the backend. The code was not written with that in mind, and it caused a number of bugs.

As such, I would strongly recommend that before a cross-fade is implemented, the concepts of "effects" and "transitions" are implemented clearly and separately from eachother:

• An effect would take one or more input video streams (frames?) and produce one output stream (frame?). An effect could be a "null" effect, which would just reference the input stream (frame?) as the output.
• A transition would take one or more input effects, and gradually transition from one effect to the other.

[MaZderMind]

@yoe thank you very much for your in-depth insight into the chalenges you were facing with DVSwitch. It is deeply welcomed and will probably save us some headache once we're coming around to implement transitions.

[yoe]

On Mon, Sep 18, 2017 at 09:26:39AM +0000, Peter wrote:

@yoe thank you very much for your in-depth insight into the chalenges you were facing with DVSwitch. It is deeply welcomed and will probably save us some headache once we're coming around to implement transitions.

Any time.

Rereading what I wrote last week, I notice that there's one thing missing that I had wanted to mention, too:

DVswitch used radiobutton-like controls to show which source was active. That worked fine as long as there were no transitions; but as soon as there were, during the transition the radiobutton was either still showing the old source as active (wrong), or showing the new source as already active (also wrong). The right thing to do would have been to drop the radiobuttons, make them regular punchbuttons, and to have another control showing that the source was active in some way (e.g., a red/blue ball[1] could show that the source is active, and its color could gradually migrate from red to blue as the transition is progressing.

Since voctomix uses a similar UI paradigma, rethinking that might be a good idea in the long run.

[1] not red/green, as some people can't make out the difference.

-- Could you people please use IRC like normal people?!?

-- Amaya Rodrigo Sastre, trying to quiet down the buzz in the DebConf 2008 Hacklab

[fightling]

I tired to play a little with transition of side-by-side view:

using this code:

#!/usr/bin/env python3
import Image
import ImageDraw
import math
import time

def timing(s):
    """ timing is a sinus wave
    """
    t = (math.cos((s - 1.0) * math.pi) + 1.0) / 2.0
    print ("timing {} => {}", s, t)
    return t

def reciprocal_distorsion(s):
    """ reciprocal distorsion (sources overlap the whole time)
    """
    return 2.0 * (1 - 1.0 / (s + 1))

def cos_distorsion(s):
    """ cosinus distorsion (sources overlap a short time)
    """
    return 1 - math.cos(s * (math.pi / 2.0))

def transit(_from, _to, _timing, _distorsion):
    """ move x,y coordinates with timing an ratio distorsion
    """
    return (_from[0] + float(_to[0] - _from[0]) * _timing * _distorsion,
            _from[1] + float(_to[1] - _from[1]) * _timing)

def transitrect(_from, _to, _timing, _distorsion):
    """ move rectangle coordinates with timing an ratio distorsion
    """
    tl = transit((_from[0], _from[1]), (_to[0], _to[1]), _timing, _distorsion)
    wh = transit((_from[2] - _from[0], _from[3] - _from[1]),
                 (_to[2] - _to[0], _to[3] - _to[1]), _timing, 1)
    return (tl[0], tl[1], tl[0] + wh[0], tl[1] + wh[1])

def overlap(A, B):
    """ check if two rectangles overlap
    """
    return A[0] < B[2] and A[2] > B[0] and A[1] < B[3] and A[3] > B[1]

def animate_transition(size, A, B, filename, stups=50):
    """ create transition animation into several files
    """
    # do 50 frames
    steps = 50
    # create an image to draw into
    image = Image.new('RGBA', size, (0, 0, 0, 0))
    # create a drawing context
    draw = ImageDraw.Draw(image)
    # if true flip sources
    flip = False
    # do all images
    for i in range(0, steps + 1):
        # get a 0.0 .. 1.0 for our progress
        progress = float(i) / steps
        # clear backgorund of that image
        draw.rectangle((0, 0, size[0], size[1]), fill=(0, 0, 0))
        # move both rectangles to there new position
        rA = transitrect(A, B, timing(progress), cos_distorsion(progress))
        rB = transitrect(B, A, timing(progress), cos_distorsion(progress))
        # check if we end the sequence or both rectangles left overlapping
        if i == steps or not (flip or overlap(rA, rB)):
            # flip sources
            flip = True
        # draw rectangles as source A/B or B/A
        if flip:
            draw.rectangle(rB, fill=(128, 0, 0))
            draw.rectangle(rA, fill=(0, 0, 128))
        else:
            draw.rectangle(rA, fill=(128, 0, 0))
            draw.rectangle(rB, fill=(0, 0, 128))
        # save an image
        image.save(filename + "%02d.png" % i)

# create animation for picture in picture
animate_transition((960, 540), (0, 0, 800, 800 / 1.77777),
                   (960 - 200, 540 - 200 / 1.77777, 960, 540), "sidebyside-preview")
animate_transition((960, 540), (0, 120, 480, 120 + 270),
                   (480, 120, 960, 120 + 270), "sidebyside-equal")

Just an experiment to get an impression of the problem!

[MaZderMind]

Looks interesting! Just one hint (don't know if you already figured that out): GStreamer can run Abimations through the use of GObject's times parameter bindings: https://github.com/MaZderMind/intro-outro-fade/blob/master/intro-outro-fade.py#L67

[fightling]

Today I continued to get some more flexible animations. For example PiP was a problem when using simple sinus or reciprocal functions for the motion. But it just looked good at first because it wouldn't need extra configuration.

The last two days I tried another approach which uses B-Splines to interpolate all four corners of the sources to their destination points. The current algorithm is doing that interpolation with scipy and numpy. It then picks the best (less weird) curve for both sources, creates an equal distribution on that curve and then animates these two corners and the width (and height) of the images from start to end. The input for the algorithm are n>2 boxes which describe start and end point and some boxes to cross in between. So you give it a list of boxes and it interpolates between those.

The current demo uses 3 boxes for A and 3 for B. The middle boxes are next to each other without overlapping which causes the animation routine to switch A/B (red/blue) to toggle the z-layer which is necessary for a good effect.

What's missing is the sinus wave to accelerate and decelerate nicely (which was already implemented in the last example). I will add that hopefully tomorrow (aka today). Then I think we have a good and flexible animation generator with an easy configuration.

Here is my experimental code:

#!/usr/bin/env python3
from PIL import Image
from PIL import ImageDraw
import math

from scipy import spatial
from scipy import interpolate as spi
import numpy as np
import sys

def makerect(x, y, w, r=1.77777):
    """ create a rectangle at x,y with a width of w and a aspect ratio of r
    """
    return (x, y, x + w, y + w / r)

def drawpoint(draw, x, y, color, radius=2):
    """ paint a point on context draw at x,y with color and radius
    """
    draw.rectangle((x - radius, y - radius, x +
                    radius, y + radius), fill=color)

def turns(array):
    """ return True if values in array has a turning point (maximum or minimum)
    """
    t = False
    less = None
    for i in range(1, len(array)):
        if less in [None, True] and array[i - 1] > array[i]:
            if less == None:
                less = False
            else:
                t = True
        if less in [None, False] and array[i - 1] < array[i]:
            if less == None:
                less = True
            else:
                t = True
    return t

def overlap(A, B):
    """ check if two rectangles A and B overlap
    """
    return A[0] < B[2] and A[2] > B[0] and A[1] < B[3] and A[3] > B[1]

def find_index_of_nearest_xy(xa, ya, x, y):
    """ find the point in xa,ya which is nearest to x,y
    """
    distance = (xa - x)**2 + (ya - y)**2
    idx = np.where(distance == distance.min())
    return idx[0][0]

def interpolate(points):
    """ do a B-Spline interpolation between the given points
        returns interpolated points and points which are nearest to input points
    """
    # re-arrange points
    data = np.array(points)
    # do interpolation
    tck, u = spi.splprep(data.transpose(), s=0, k=2)
    unew = np.arange(0, 1.001, 0.0001)
    out = spi.splev(unew, tck)
    # skip result if new curve is too wild
    if (turns(out[0]) and turns(out[1])):
        return [[], []], []
    # find and draw matching points
    nearest = []
    for p in points:
        i = find_index_of_nearest_xy(out[0], out[1], p[0], p[1])
        nearest.append(i)
    # return interpolation and nearest points
    return out, nearest

def dointerp(draw, rects, x, y, f):
    # get corner defined by x,y from rectangles
    corners = []
    for i in rects:
        corners.append((i[x], i[y]))
    # interpolate between corner points
    o, c = interpolate(corners)
    print(c)

    # skip if we got no interpolation
    if not (o and c):
        return [], []

    # create uniform distribution of points between the corner points
    # with same amount of points between every corner
    for j in range(0, 7):
        b = []
        b.append(c[0])
        for i in range(1, len(c)):
            b.append(c[i - 1] + (c[i] - c[i - 1]) / 2)
            b.append(c[i])
        c = b

    ws = rects[0][2] - rects[0][0]
    we = rects[-1][2] - rects[-1][0]
    hs = rects[0][3] - rects[0][1]
    he = rects[-1][3] - rects[-1][1]
    r = []
    j = 0
    for i in c:
        w = ws + (we - ws) * j / len(c)
        h = hs + (he - hs) * j / len(c)
        if x == 0:
            if y == 1:
                r.append((o[0][i], o[1][i], o[0][i] + w, o[1][i] + h))
            else:
                r.append((o[0][i], o[1][i] - h, o[0][i] + w, o[1][i]))
        if x == 2:
            if y == 1:
                r.append((o[0][i] - w, o[1][i], o[0][i], o[1][i] + h))
            else:
                r.append((o[0][i] - w, o[1][i] - h, o[0][i], o[1][i]))
        drawpoint(draw, o[0][i], o[1][i], f, 1)
        j += 1
    return r, c

def best(aa):
    mind = sys.maxint
    b = []
    for j in range(0, len(aa)):
        a = aa[j][1]
        if len(a) > 1:
            min = sys.maxint
            max = 0
            for i in range(1, len(a)):
                if abs(a[i] - a[i - 1]) < min:
                    min = a[i] - a[i - 1]
                if abs(a[i] - a[i - 1]) > max:
                    max = a[i] - a[i - 1]
            if max - min < mind:
                mind = max - min
                b = aa[j][0]
    return b

def animate(d, a, f):
    r = []
    r.append(dointerp(d, a, 0, 1, f))
    r.append(dointerp(d, a, 2, 1, f))
    r.append(dointerp(d, a, 0, 3, f))
    r.append(dointerp(d, a, 2, 3, f))
    return best(r)

# canvas size
size = (960, 540)

# create list of rectangles
A = []
B = []
# PiP scenario
A.append(makerect(0, 0, 960))
B.append(makerect(800, 540 - 94, 150))

# side by side
A.append(makerect(10, 135, 460))
B.append(makerect(490, 135, 460))

# one above the other
# A.append(makerect(240, 0, 460))
# B.append(makerect(240, 270, 460))

# swapped PiP scenario
A.append(B[0])
B.append(A[0])

# create an image to draw into
image = Image.new('RGBA', size, (0, 0, 0, 0))
# create a drawing context
draw = ImageDraw.Draw(image)
# clear backgorund of that image
draw.rectangle((0, 0, size[0], size[1]), fill=(0, 0, 0))

draw.rectangle(A[0], fill=(128, 0, 0))
draw.rectangle(A[1], fill=(256, 0, 0))
draw.rectangle(B[1], fill=(0, 0, 256))
draw.rectangle(B[0], fill=(0, 0, 128))

print "calculating animation..."
Ac = animate(draw, A, (256, 128, 128))
Bc = animate(draw, B, (128, 128, 256))

# for i in range(0, len(x) - 1):
#    drawpoint(draw, x[i], y[i], (128, 128, 0))

print "saving images..."
image.save("analysis-cubic.png")
flip = False
for i in range(0, len(Ac)):
    # clear backgorund of that image
    draw.rectangle((0, 0, size[0], size[1]), fill=(0, 0, 0))
    if i == len(Ac) - 1 or not (flip or overlap(Ac[i], Bc[i])):
        # flip sources
        flip = True
    if flip:
        draw.rectangle(Bc[i], fill=(128, 0, 0))
        draw.rectangle(Ac[i], fill=(0, 0, 128))
    else:
        draw.rectangle(Ac[i], fill=(128, 0, 0))
        draw.rectangle(Bc[i], fill=(0, 0, 128))
    # save an image
    image.save("cubic%04d.png" % i)

There is a "# create list of rectangles" part where the configuration is done by defining two arrays of rectangles. Then the interpolation is done (still with some overhead debug code which draws the following image:

You see the big dark red background source A and small dark blue PiP source in the right bottom corner. in the middle are the side-by-side boxes in (lighter red and blue) which shall lead both boxes to a point without overlapping.

Then you can see four interpolations between start and end corners. Three for blue and one for red.

I added a not super sophisticated way to chose the "best" curve and end up with one for red and one for blue.

And this is the animation:

Super slow because there is still an fps management missing and also browsers tend to slow down fast animated GIFs. So downloading the GIF and using another viewer will fasten things up a little.

So if you like you may play around a bit with different rectangles in A[] and B[] for experimenting.

Use convert -delay 1 -loop 0 cubic*.png cubic.gifto convert the output to an animated gif.

Here is the animation using other middle rectangles (see below "# one above the other") just to give you an imagination of the flexibility:

Next steps:

• solve the sinus acceleration
• playtime configuration and FPS management
• and the last step will be to use the (pre-processable) animation with the real video sources in voctomix

[MaZderMind]

You might want to take a look at http://ftp5.gwdg.de/pub/misc/openstreetmap/FOSS4G-2016/foss4g-2016-1562-introduction_till-hd.webm where we had a video mixer which did transitions (ie minute 9:35)

[fightling]

Thx! I'll keep that in mind. Does anybody have more examples that demonstrate the demands?

[fightling]

I added the sinus wave acceleration and fps management and here are some example outputs:

(again: to see them in original speed (50 fps/1 second) try to download the files and view them locally)

sidebyside-sidebyside:

pip-sidebysidepreview:

pip-sidebyside:

pip-pip:

sidebyside-sidebysidepreview:

Next step will be to integrate it into the voctomix mixer.

[fightling]

This is the current configuration for the transition effects:

[composites]
pip               =   0,   0, 960, 540,   800, 446, 150,  84
sidebyside        =  10, 135, 460, 260,   480, 135, 460, 260
sidebysidepreview =   0,   0, 600, 353,   600, 353, 360, 203
oneupontheother   = 250,   5, 460, 260,   250, 275, 460, 260

[transitions]
pip-pip                      = pip,        sidebyside,         pip
sidebyside-sidebyside        = sidebyside, oneupontheother,    sidebyside
pip-sidebyside               = pip,        sidebyside
pip-sidebysidepreview        = pip,        sidebysidepreview
sidebyside-sidebysidepreview = sidebyside, sidebysidepreview

In the section composites you can freely name and add scenarios which consist of two rectangles (x_A,y_A,w_A,h_A, x_B,y_B,w_B,h_B). In the second section called transitions one can define a list of those composites to define the animations.

I think this is easy to use and quite flexible.

[MaZderMind]

@fightling I really don't like the list of meaningless numbers. Even I don't know what they stand for and someone new to the project can't figure them out by just looking at them.

Better split them into different sections and name them:

[composite]
composites=pip,sidebyside,lalal

[composite.pip]
a-width=123
a-left=456
…

What does defining a transition "pip, sidebyside, pi" mean? What would happen If I don't define it?

In my imagination, switching from pip to sidebyside would now result in a animation, instead of a cut. What role do the defined (or not defined) transition-sequences play here?

When working on this, keep the crop-Functionality in mind. Sometimes the B-Picture is cropped to be a square, while the A-source is still 16:9. Just a thing to keep in mind.

[fightling]

I really don't like the list of meaningless numbers. Even I don't know what they stand for and someone new to the project can't figure them out by just looking at them.

Hm... I don't like to unnecessarily stretching things up over too many lines. But let's give it a try:

[composite.pip]
a.x = 0
a.y = 0
a.width = 960
a.height = 540
b.x = 800
b.y = 446
b.width = 150
b.height = 84

[composite.sidebyside]
a.x = 10
a.y = 135
a.width = 460
a.height = 260
b.x = 480
b.y = 135
b.width = 460
b.height = 260

[composite.sidebysidepreview]
a.x = 0
a.y = 0
a.width = 600
a.height = 353
b.x = 600
b.y = 353
b.width = 360
b.height = 203

[composite.oneupontheother]   
a.x = 250
a.y = 5
a.width = 460
a.height = 260
b.x = 250
b.y = 275
b.width = 460
b.height = 260

[transitions]
pip-pip                      = pip,        sidebyside,         pip
sidebyside-sidebyside        = sidebyside, oneupontheother,    sidebyside
pip-sidebyside               = pip,        sidebyside
pip-sidebysidepreview        = pip,        sidebysidepreview
sidebyside-sidebysidepreview = sidebyside, sidebysidepreview

What does defining a transition "pip, sidebyside, pi" mean? What would happen If I don't define it?

pip-pip = pip, sidebyside, pip

This line defines a transition from PIP to PIP with switched sources. So the PIP is the start composite and the transition effect is a morph from PIP to side-by-side to PIP again. While the transition the sources will be swapped automatically (and so the z-layer will be) at the first point where they are not overlapping. The name pip-pip is meaningless and just there for readability. You could also name it 1 or lala without any effect.

pip-sidebyside = pip, sidebyside

This transition is from pip to side-by-side without any composite in the middle.

You could also define more complex transitions with more than three composites but I didn't do any tests for this.

In my imagination, switching from pip to sidebyside would now result in a animation, instead of a cut. What role do the defined (or not defined) transition-sequences play here?

I would like to select a matching transition automatically by comparing source and destination composite. The pip-pip transition would be used when you switch sources A and B while in PIP. Other transitions like pip-sidebyside would be used when you switch from PIP to side-by-side view.

Currently I would use transitions whenever they are defined. If there is no transition for an action I would just do a cut. Another possibility would be to give the control about when to use an available transition to the user. For that we need to change the UI somehow. Like a checkbox for using transitions or not.

When working on this, keep the crop-Functionality in mind. Sometimes the B-Picture is cropped to be a square, while the A-source is still 16:9. Just a thing to keep in mind.

I have that in mind and I think we can add cropping to the composites configuration. For example like this:

[crop.sidebyside]
b.left = 210
b.right = 210

To crop 210 pixels from b in side-by-side view. b.top and b.bottom would be 0 by default. Also a.left, a.top, a.bottom and a.right would be 0 by default. The composite then needs to have the corresponding proportions. I would suggest to auto-crop or enhance the source automatically until it fits into the composite rectangles.

I know it's getting complicated a bit. But I put a lot of algorithms into the code to enable an easy way to define animations by just defining a small number of rectangles. I think this is already very handy and could be much more complicated if you think about it.

Defining composites is just a little more explicit than setting up views in the older configuration but much more capable! Usually transitions can be left untouched or commented out if you don't want to use them and the user will just need to changed the composite coordinates which is quite similar to what he did before.

[fightling]

I'm nearly finished with the transition animations. Here are some new features:

• converted all standard voctomix composites
• automatic transition selection
• alpha blending
• intermediate composites enhance possibilities
• FPS management (transitions have individual durations which will be automatically converted into a number of frames needed)
• improved configuration (proportional coordinates for example to easily use the same configuration with different resolutions without changing the transition configuration!)
• test application now has several command line options

open issues are:

• cropping (already in configuration but not processed)
• default sources (already in configuration but not processed)
• use animations in voctocore by reconfiguring mixing pipeline continuously
• selectable transition option in voctogui

I have made a video on youtube with 16 concatenated transitions which are automatically selected by using 4 target composites. And 6 intermediate composites to make the animations more fancy.

I did my very best to remain as most compatible as possible to the old composite definition capabilities and I think this demo shows that the new configuration format can cover all former options and brings a lot of new features and much more flexibility with just a tiny amount of more complexity. If you take a deeper look into the configuration you may get adapted quickly.

Here is the configuration which was used to make the demo video:

[composites]
; List of configurations of custom named composites for mixing video sources A and B.
;
;   attribute                format         default
;   NAME.a                 = RECT         ; no output
;   NAME.b                 = RECT         ; no output
;   NAME.crop              = CROP         ; no cropping
;   NAME.crop-a            = CROP         ; no cropping
;   NAME.crop-b            = CROP         ; no cropping
;   NAME.default-a         = INPUT        ; do not change source
;   NAME.default-b         = INPUT        ; do not change source
;   NAME.alpha-a           = ALPHA        ; opaque
;   NAME.alpha-b           = ALPHA        ; opaque
;   NAME.inter             = BOOL         ; not intermediate
;
; NAME = unique composite name
;
; RECT = Rectangular coordinates which are given by
;
;       X/Y WxH
;   or  POS WxH
;   or  X/Y SIZE
;   or  POS SIZE
;   or  *
;
;   X,Y,W,H can be mixed integer absolute coordinates or float proportions
;   POS and SIZE must both be float proportions
;   * stands for full screen size (0/0 1.0x1.0)
;
; CROP = Cropping borders which are given by
;
;       L/T/R/B
;
;   L,T,R,B can be mixed integer absolute coordinates or float proportions
;
; INPUT = Any available input source name (grabber, cam1, cam2, ...)
;
; ALPHA = numeric value in the range between 0 (invisible) and 255 (opaque) or
;         float value between 0.0 (invisible) and 1.0 (opaque)
;
; BOOL = some value. if non-empty the option will be set ('true' for example) 

; fullscreen
fullscreen.a                        = *

; fullscreen-1
fullscreen-1.a                      = *
fullscreen-1.b                      = *
fullscreen-1.alpha-b                = 0.0
fullscreen-1.inter                  = true

; fullscreen-2
fullscreen-2.a                      = *
fullscreen-2.b                      = *
fullscreen-2.inter                  = true

; fullscreen-pip
fullscreen-pip.a                    = *
fullscreen-pip.b                    = 1741/979 0.0
fullscreen-pip.alpha-b              = 0.0
fullscreen-pip.inter                = true

; fullscreen-sidebysidepreview
fullscreen-sidebysidepreview.a      = *
fullscreen-sidebysidepreview.b      = 1.0 0.0
fullscreen-sidebysidepreview.alpha-b= 0.0
fullscreen-sidebysidepreview.inter  = true

; fullscreen-sidebysidepreview
fullscreen-sidebyside.a             = *
fullscreen-sidebyside.b             = 1.0/0.5 0.0
fullscreen-sidebyside.alpha-b       = 0.0
fullscreen-sidebyside.inter         = 0.0

; PiP
pip.a                               = *
pip.b                               = 0.82 0.16
pip.crop                            = 0/600/0/600
pip.default-a                       = grabber
pip.default-b                       = cam1

; side-by-side
sidebyside.a                        = 5/273 950x534
sidebyside.b                        = 965/273 950x534
sidebyside.default-a                = cam1
sidebyside.default-b                = cam2

; side-by-side preview
sidebysidepreview.a                 = 12/12 0.75
sidebysidepreview.b                 = 0.74 0.25
sidebysidepreview.crop-b            = 0/640/0/640
sidebysidepreview.default-a         = grabber
sidebysidepreview.default-b         = cam1

# one-opon-the-other (like side-by-side but vertical)
oneupontheother.a                   = 485/4 950x534
oneupontheother.b                   = 485/542 950x534
oneupontheother.inter               = true

[transitions]
; list of transitions each one can be freely named and is a list of composites
; which will be morphed into an animation. Interpolation will be linear with two
; composites and B-Splines for more.

; unique name                       =   ms, from / [... /] to
pip-pip                             = 1000, pip / sidebyside / pip
sidebyside-sidebyside               = 1000, sidebyside / oneupontheother / sidebyside
pip-sidebyside                      = 1000, pip / sidebyside
pip-sidebysidepreview               = 1000, pip / sidebysidepreview
sidebyside-sidebysidepreview        = 1000, sidebyside / sidebysidepreview
sidebysidepreview-sidebysidepreview = 1000, sidebysidepreview / sidebyside / sidebysidepreview
fullscreen-pip                      = 1000, fullscreen-pip / pip
fullscreen-sidebyside               = 1000, fullscreen-sidebyside / sidebyside
fullscreen-sidebysidepreview        = 1000, fullscreen-sidebysidepreview / sidebysidepreview
fullscreen-fullscreen               = 1000, fullscreen-1 / fullscreen-2

As you might see there are several intermediate composites (marked by the *.inter flag). One is called onupontheother and is just used for an intermediate step within the sidebyside-sidebyside transition to make the pictures move around each other.

The fullscreen-* composites are also intermediate and some are using alpha fading (for transition between fullscreen to itself for example). Others are used to fade between fullscreen and pip, sidebyside or sidebysidepreview which optimizes the way these transitions look like.

There are so much more possibilities! After some further testing I will push my experimental branch and you will be invited to play around by yourself.

[fightling]

@derpeter @MaZderMind cropping is not functional in voctomix or is it?

The only code I could find is in file videomix.py:

        try:
            pipcrop = [int(i) for i in Config.get('picture-in-picture',
                                                  'pipcrop').split('/', 3)]
            self.log.debug('PIP-Video-Cropping configured to %u/%u/%u/%u',
                           pipcrop[0], pipcrop[1], pipcrop[2], pipcrop[3])
        except NoOptionError:
            pipcrop = [0, 0, 0, 0]
            self.log.debug('PIP-Video-Cropping calculated to %u/%u/%u/%u',
                           pipcrop[0], pipcrop[1], pipcrop[2], pipcrop[3])

Except logging this does not seem to have any effect. But I'm reworking it anyway. So my real question is: How are the numbers in the voctocore/default-config.ini at pipcrop meant?

[picture-in-picture]
;pipsize = 320x180
;pipcrop=0/600/0/600
;pippos = 948/528

To start a proper discussion, I would like to offer my favorite: I would like to use pixel borders (left,top,right,bottom) within the source frame (related to it's original resolution). So you first cut the borders of the source image and then it get's resized within the animation. Also it would be possible to use float numbers for proportional values like my rectangle configuration.

But 0/600/0/600 makes no sense with that interpretation. Please tell me, what you need!

Another hint about what I'm currently trying is this animated GIF:

Open this in full screen because the cropped blue box' original size is drawn as a thin blue line surrounding the blue box.

Here is an update of the current transitions. some changes when pip is involved and sidebyside-sidebyside is a little more fancy then before.

[fightling]

It seems like acrop and bcrop within the side-by-side-preview configuration is working. And like in this code...

        try:
            bcrop = [int(i) for i in Config.get('side-by-side-preview',
                                                'bcrop').split('/', 3)]
            self.log.debug('B-Video-Cropping configured to %u/%u/%u/%u',
                           bcrop[0], bcrop[1], bcrop[2], bcrop[3])
        except NoOptionError:
            bcrop = [0, 0, 0, 0]
            self.log.debug('B-Video-Cropping calculated to %u/%u/%u/%u',
                           bcrop[0], bcrop[1], bcrop[2], bcrop[3])

...and this...

            if idx == self.sourceA:
                pad.xpos, pad.ypos = apos
                pad.croptop, \
                    pad.cropleft, \
                    pad.cropbottom, \
                    pad.cropright = acrop
                pad.width, pad.height = asize
                pad.zorder = 1  

...it seems the order of the crop values is: top, left, bottom, right instead of - what I would expect - left, top, right, bottom. Is that intended?

But this explains what 0/600/0/600 means: A cropping of left and right border of 600 pixels (which must be related to the input size 1920x1080).

[fightling]

I've added another video which tries to simulate the effect originally wished in issue #179.

But I don't like the configuration:

sidebysidepreview.a                 = 12/12 0.75
sidebysidepreview.b                 = 0.60/0.42 0.56
sidebysidepreview.crop-b            = 600/0/600/0
sidebysidepreview.default-a         = grabber
sidebysidepreview.default-b         = cam1

...because the coordinates of sidebysidepreview.b are meant to read as the original (un-cropped) image's position. Wouldn't it be easier to set the position of the cropped image - but what's about the width and height in sidebysidepreview.b then? They must have the cropped proportions then - ugly in either way. Some help?

P.S.: sidebysidepreview.crop-b is meant to be left,top,right,bottom here!

[derpeter]

@fightling i agree that the current way to configure is not very intuitive. At fusion it has taken 3 people and 2 hours to configure what we want to see ;-). IMHO you suggested is more intuitive. I don't see, in our use case, situations where we would crop asymmetric. Therefore a target pixel width / height would be enough. The not specified value should be calculated to keep the aspect ratio. But there maybe use case i don't see where some would like to crop asymmetric. In that case i would also go for a configuration where you specify either crop left+right or top+bottom and calculate the rest to keep the aspect.

Regarding positioning i would go for xy coordinates of the upper left corner of the cropped image

[fightling]

Thank you for your evaluation, @derpeter. We should make coordinate formats clear early because I just uploaded a repository with the code and documentation to https://github.com/fightling/voctomix-transitions. Look into https://github.com/fightling/voctomix-transitions/blob/master/README.md to get further information.

I hope it was a good idea to make a github repo first and later move the code into a new branch of voctomix for integration.

[computersrmyfriends]

@fightling How do I use what you have done along with Voctocore?

[fightling]

After I've integrated it. There is some research to do before. That's why I've started a separate repo. So you might see a pull request when the time has come.

[fightling]

I've pushed the branch https://github.com/voc/voctomix/tree/feature/transitions where you can see the working transitions. It's still work in progress, but you might get an impression of how transitions work. I know that a lot of UI actions do not lead to the correct transitions right now. I'm currently working on a slightly different UI concept that uses the new possibilities.

[computersrmyfriends]

How do I use this through voctocore?

[computersrmyfriends]

Is it integrated with Voctocore?

[fightling]

It's integrated now but not bug free. There are several issues left to solve but you may checkout where it's going. BUT DON'T USE IT IN PRODUCTION. I'll try to solve the remaining issues in the upcoming week and will write a message here. You will find the composite and transition configuration at the end of the file voctocore/default-config.ini like I've described in https://github.com/fightling/voctomix-transitions/blob/master/README.md.

[computersrmyfriends]

Wow great work so far.

[computersrmyfriends]

This can be used only through code? Can it be used on the commamdline like voctocore?

[fightling]

This will be integrated into voctocore. You do not need to write code. There will be a default configuration and you will be able to customize that configuration as described. The explanations about the code are just for internal documentation.

[computersrmyfriends]

Any idea when it will be integreated into voctocore so that I can try it on the command line?

Thanks

[fightling]

Here is a new video showing the advanced transition capabilities and the new GUI of voctomix2:

https://www.youtube.com/watch?v=U4sBNQHvzvs&feature=youtu.be