External media support - static images, video containers, etc.

[wobbals]

Proposal: extend manifest files to allow image inputs like:

{
   "createdAt" : 16918094607,
   "files" : [
      {
         "filename" : "slide1.png",
         "startTimeOffset" : 562,
         "stopTimeOffset" : 373156,
         "layoutClass" : "focus",
         "streamId" : "someuniqueidentifier" // See note below
      }
  ]
}

As far as format restrictions goes, I can't promise anything without doing the implementation itself, but we'll most likely continue to use magickcore for the provider, so anything that can be taken by MagickReadImageFile should be acceptable.

For streamId: In order for this to work with https://gist.github.com/wobbals/15f2876d9fd49ce2fdf3e9046adabb2f , we need a unique identifier. Even though streamId doesn't really make sense, using the same key allows for consistent treatment with the streamUpdated layout event.

[wobbals]

@robinlambert @Tom-Tom wdyt?

[robinlambert]

Looks sweet! Will it be possible to set the object-fit for those external medias? Since it will mostly be slides or videos, we would like to keep the aspect ratio and the object-fit in contain.

Also, for video medias it's pretty complex in the context of our app because the user is able to pause de video or to go forward and backward in it. I don't know how you would handle such a behaviour. Maybe the easiest way would be to include a videoStartTime in the file object, so that when the user go forward in the video, we create a new file object with the new videoStartTime. I don't know if it make sense.

[wobbals]

Object fit is always available to the CSS stylesheet for any element. In your case, I think you just need to modify the presets (see Geometry.h) and pass the stylesheet as a custom. For example:

./barc
-pcustom
-c "stream { \
    float:left; \
    margin-top: 80%; \
width: 20%; \
height: 20%; \
object-fit: contain; \
} \
stream.focus { \
position: absolute; \
top: 0; \
left: 0; \
margin-top: 0px; \
height: 80%; \
width: 100%; \
object-fit: contain; \
}"

If you're passing stylesheets from any JSON context (webapp input parameters or manifest), encode the custom stylesheet as a single-line string attribute. This already works today for the layoutEvents:

"layout":
       {
         "type": "custom",
         "stylesheet": "stream { float:left; width: 10%; height: 20%; object-fit: contain; } ..."
       }

[wobbals]

I'm not following your point about scrubbing video forward and backward. Are you suggesting to support seek/scrub events during the output sequence of some audio/video media source?

[robinlambert]

Yes, it would be great as we support this during live in our app.

[wobbals]

I think we'll need more clarification on the scrubbing issue, let's deal with that separately. For now, I think there's enough to go on for static image content on this issue, so I'll plan to do this after the cluster management work is completed.