Closed slifty closed 3 years ago
A segment is a time-constrained subsection of a source. For instance, a source might have 10 hours of content and might be broken into twenty separate 30 minute segments.
Right now the position
attribute of a payloads is based on how many milliseconds into the video source
a payload is.
With segments, we might have position
instead represent how many milliseconds into the segment
a payload is. This means a given source could produce multiple STREAM.CONTAINER
payloads with position 0
(one per segment). It essentially breaks a single source into multiple "videos."
Alternatively, we might create a new type of SEGMENT
payload (which, as a payload, would have a position as well). This would allow a given appliance / application that cares about segments to calculate "segment position" -- keeping track of new segment events and doing simple math (e.g. position - segment start position = segment position
).
A segment might also have metadata associated with it (e.g. program name, host name, program description, etc). We have a vague concept of "metadata" associated with payloads, but I am worried that adding these to the payloads themselves would quickly make things very heavy (e.g. if the program name is included in every single payload that would dramatically increase the size of a payload).
As an aside: the timestamp
of a given payload would still exist outside of the context of the segment -- timestamp is defined as being absolute time, so it has nothing to do with segments.
I'm getting convinced that segmentation (A) should be explicit and (B) should NOT modify the positions of payloads directly (or any aspect of a payload). Rather, a segment should be conveyed via its own payload.
This would be more robust and also potentially allows multiple types of segmentation to exist in parallel on a single stream. The tradeoff is that segment context must be "remembered" by applications and appliances that care about segmentation.
For my immediate use case, time is the driver of segments: e.g. I have a schedule of programs for a channel in a given day, and I know that there is a new program at the top of an hour / every 30 minutes / etc. (I may know the names of those programs as well)
For pre-recorded video, there may be a list of specific segment times (e.g. we know when each question was read in a debate and want to segment the debate according to those times).
segmentor
appliance that could initiate new segments based on stream-processing logic (e.g. in response to payload data in addition to payload timestamp).It seems like the answer to all of these things is... yes!
Should we have a SEGMENT.START
payload (should we have a SEGMENT.END
payload) <-- It would be more apt to find a term that captures a generic delimiter (e.g. SEGMENT.MARK
or SEGMENT.DELIMITER
). TBD.
Appliances that care about segmentation can consume that payload type. Appliances that don't care about segmentation can just go about their business.
We might eventually want the countertop to be able to dynamically turn appliances on and off -- but that's deeper and doesn't have to be coupled to segmentation directly (indeed, the logic there could be based on payload analysis, and if segments are payloads... ALL THE BETTER)
Good talk everybody.
Discussion
What do you want to talk about?
Let's talk about segmentation.
A video source might have distinct content segments that should be treated as contextually coherent "sets" of data, potentially with its own metadata that applies to all payloads within it.
For example: TV stations have programs that run for a specific duration of time and have explicit metadata associated (e.g. program name, hosts, etc.).
what is a "segment" at a technical level? How does a "segment" manifest in a given payload (if it does at all)?
Once answer these things, we need to decide where in our architecture segmentation should actually occur. What can trigger the creation of a new segment? What should happen when a new segment is created?
Other Thoughts
To complicate things, within a segment there may be sub-segments -- for instance, individual commercials could be segments. Would the ideal segmentation architecture allow for a TVK implementation to dynamically initiate new segments?
Relevant Resources / Research
We started to explore some of these things over in the appliances repository -- inspired by the SRT generation appliance.
https://github.com/tvkitchen/appliances/issues/70