Add support for parts CR 10-19

jolla56 commented 5 years ago

Description of the change: (free-text)

We should be able to convey part metadata via ninjs.

In NewsML-G2, items can have many wrappers, each expressing properties for an identifiably separate part of the content; for example each of the shots, or segments, which make up a video.

Each part should have properties for:

an ID for the part
a sequence number
an icon that may help to visually identify the content of the part
the start time and duration of the segment
And all of the other ninjs properties, except for renditions and associations.

bquinn commented 4 years ago

To compare with partMeta in NewsML-G2 / IPTC News Architecture:

Here's a multi-part video example:

<?xml version="1.0" encoding="ISO-8859-1"?>
<newsItem
    xmlns="http://iptc.org/std/nar/2006-10-01/"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://iptc.org/std/nar/2006-10-01/
        ./NewsML-G2_2.28-spec-All-Power.xsd"
    standardversion="2.28"
    guid="tag:example.com,2008:407624"
    version="12"
    standard="NewsML-G2"
    conformance="power"
    xml:lang="en">
    <catalogRef
        href="http://www.iptc.org/std/catalog/catalog.IPTC-G2-Standards_34.xml" />
    <catalogRef
        href="http://www.example.com/metadata/newsml-g2/catalog.NewsML-G2.xml" />
    <rightsInfo>
        <usageTerms>
            Access only for Eurovision Members and EVN / EVS Sub-Licensees.
            <br />
            Coverage cannot be used by a national competitor of the contributing
            broadcaster.
        </usageTerms>
    </rightsInfo>
    <itemMeta>
        <itemClass qcode="ninat:video" />
        <provider qcode="providercode:EBU">
            <name>European Broadcasting Union - EVN</name>
            <organisationDetails>
                <contactInfo>
                   <web>http://www.eurovision.net</web>
                   <phone>+41 22 717 2869</phone>
                   <email>features@eurovision.net</email>
                   <address role="AddressType:Office">
                       <line>Eurovision Sports News Exchanges</line>
                       <line>L Ancienne Route 17 A</line>
                       <line>CH-1218</line>
                       <locality>
                           <name>Grand-Saconnex</name>
                       </locality>
                       <country qcode="ISOCountryCode:ch">
                           <name xml:lang="en">Switzerland</name>
                       </country>
                   </address>
                </contactInfo>
            </organisationDetails>
        </provider>
        <versionCreated>2019-10-19T10:54:04Z</versionCreated>
        <firstCreated>2008-11-06T10:22:28Z</firstCreated>
        <pubStatus qcode="stat:usable" />
        <service qcode="servicecode:EUROVISION">
            <name>Eurovision services</name>
        </service>
        <edNote>Originally broadcast in Germany</edNote>
        <link rel="irel:associatedWith"
            href="http://www.example.com/video/407624/index.html"/>
    </itemMeta>
    <contentMeta>
        <contentCreated> 2008-11-05T19:04:00-08:00</contentCreated>
        <located type="cptype:city" qcode="city:345678">
            <name>Berlin</name>
            <broader type="cptype:statprov" qcode="state:2365">
                <name>Berlin</name>
            </broader>
            <broader type="cptype:country" qcode="iso3166-1a2:DE">
                <name>Germany</name>
            </broader>
        </located>
        <creator qcode="codesource:DEZDF">
            <name>Zweites Deutsches Fernsehen</name>
            <organisationDetails>
                <location>
                   <name>MAINZ</name>
                </location>
            </organisationDetails>
        </creator>
        <contributor qcode="codeorigin:DEZDF" role="rolecode:TechnicalOrigin">
            <name>Zweites Deutsches Fernsehen</name>
        </contributor>
        <creator qcode="codesource:GBRTV">
            <name>Reuters Television Ltd</name>
        </creator>
        <language tag="en" role="langusecode:VoiceOver" >
            <name>English</name>
        </language>
        <genre qcode="genre:biog">
            <name xml:lang="en-GB">Biography</name>
            <name xml:lang="fr">biographie</name>
        </genre>
        <subject type="cpnat:abstract" qcode="medtop:01000000">
            <name xml:lang="en-GB">Arts, Culture and Entertainment</name>
            <name xml:lang="fr">Arts, culture, et spectacles</name>
            <narrower type="cpnat:abstract" qcode="medtop:20000003">
                <name xml:lang="en-GB">Animation</name>
                <name xml:lang="fr">Dessin animé</name>
            </narrower>
        </subject>
        <headline>Loriot retrospective</headline>
        <description role="descrole:dopesheet">
            Yesterday evening (November 5) an exhibition opened in Berlin in
            honour of German humorist Vicco von Bülow, better known under the
            pseudonym "Loriot", to commemorate his 85th birthday. He was born
            November 12, 1923 in Brandenburg an der Havel and comes from an old
            German aristocratic family. He is most well-known for his cartoons,
            television sketches alongside late German actress Evelyn Hammann and
            a couple of movies. Under the name "Loriot" in 1971 he created a
            cartoon dog named "Wum", which he voice acted himself. In 1976 the
            first episode of the TV series "Loriot" was produced.
            <br />
        </description>
        <description role="descrole:shotlist">
            Berlin, 05/11/2008
            <br />
            - vs. Vicco von Bülow entering exhibition
            <br />
            - vs. Loriot and media
            <br />
            - sot Vicco von Bülow
            <br />
            "Since 85 years I didn't succeed in pursuing a job that could be
            called a profession."
            <br />
            - vs exhibition
            <br />
            - sot Irm Herrmann, actress
            <br />
            "Loriot is timeless. You always can watch him and I can always
            laugh."
            <br />
            - actor Ulrich Matthes in exhibition
            <br />
            sot Ulrich Matthes, actor
            <br />
            " I would say: one of the great German classics. Goethe, Kleist,
            Schiller, Thomas Mann, Loriot. That's the way I would say it."
            <br />
        </description>
    </contentMeta>
    <partMeta partid="Part1_ID" seq="1">
        <icon href=" http://www.example.com/video/Keyframes/407624.jpeg"/>
        <timeDelim renditionref="vidrnd:avi25" start="0" end="446" timeunit="timeunit:editUnit"/>
        <language tag="en" role="langusecode:VoiceOver" />
        <description>Vicco von Bülow entering exhibition </description>
    </partMeta>
    <partMeta partid="Part2_ID" seq="2">
        <icon href="http://www.example.com/video/Keyframes/407624-447.jpeg"/>
        <timeDelim renditionref="vidrnd:avi25" start="446" end="831" timeunit="timeunit:editUnit"/>
        <language tag="en" role="langusecode:VoiceOver" />
        <description>Loriot and media </description>
    </partMeta>
    <partMeta partid="Part3_ID" seq="3">
        <icon href="http://www.example.com/video/Keyframes/407624-832.jpeg"/>
        <timeDelim renditionref="vidrnd:avi25" start="831" end="1081" timeunit="timeunit:editUnit"/>
        <language tag="en" role="langusecode:Interlocution" />
        <description>Vicco von Bülow interview</description>
    </partMeta>
    <partMeta partid="Part4_ID" seq="4">
        <icon href="http://www.example.com/video/Keyframes/407624-1082.jpeg"/>
        <timeDelim renditionref="vidrnd:avi25" start="1081" end="1313" timeunit="timeunit:editUnit"/>
        <language tag="en" role="langusecode:NaturalSound" />
        <description>Exhibition panorama </description>
    </partMeta>
    <partMeta partid="Part5_ID" seq="5">
        <icon href="http://www.example.com/video/Keyframes/407624-1314.jpeg"/>
        <timeDelim renditionref="vidrnd:avi25" start="1313" end="1616" timeunit="timeunit:editUnit"/>
        <language tag="en" role="langusecode:Interlocution" />
        <description>Irm Herrmann, actress, interview</description>
    </partMeta>
    <partMeta partid="Part6_ID" seq="6">
        <icon href="http://www.example.com/video/Keyframes/407624-1617.jpeg"/>
        <timeDelim renditionref="vidrnd:avi25" start="1616" end="2109" timeunit="timeunit:editUnit"/>
        <language tag="en" role="langusecode:VoiceOver" />
        <description>Ulrich Matthes, actor, in exhibition</description>
    </partMeta>
    <partMeta partid="Part7_ID" seq="7">
        <icon href="http://www.example.com/video/Keyframes/407624-2110.jpeg"/>
        <timeDelim renditionref="vidrnd:avi25" start="2109" end="2732" timeunit="timeunit:editUnit"/>
        <language tag="en" role="langusecode:Interlocution" />
        <description>Ulrich Matthes, actor, interview</description>
    </partMeta>
    <partMeta partid="Part8_ID" seq="9">
        <icon href="http://www.example.com/video/Keyframes/407624-2733.jpeg"/>
        <timeDelim renditionref="vidrnd:avi25" start="2732" end="2775" timeunit="timeunit:editUnit"/>
        <language tag="en" role="langusecode:VoiceOver" />
        <description>"I would say: one of the great German classics. Goethe, Kleist,
            Schiller, Thomas Mann, Loriot. That's the way I would say it."</description>
    </partMeta>
    <contentSet>
        <remoteContent href="http://www.example.com/video/407624.avi"
            rendition="vidrnd:avi25"
            format="fmt:avi"
        duration="111" durationunit="timeunit:seconds"
        videocodec="vcdc:c155"
        videoframerate="25"
        videoaspectratio="16:9" />
    </contentSet>
</newsItem>

bquinn commented 4 years ago

As an example,

<partMeta partid="Part1_ID" seq="1">
    <icon href=" http://www.example.com/video/Keyframes/407624.jpeg"/>
    <timeDelim renditionref="vidrnd:avi25" start="0" end="446" timeunit="timeunit:editUnit"/>
    <language tag="en" role="langusecode:VoiceOver" />
    <description>Vicco von Bülow entering exhibition </description>
</partMeta>

could become

"partmeta": [
   {
       "partid": "Part1_ID - should be a URI?",
       "sequencenumber": 1,
       "icon": {
           "href": "http://www.example.com/video/Keyframes/407624.jpeg" // could this be a rendition instead?
       },
       "timedelimiter": {
           "renditionref": "http://example.com/vidrnd/avi25", // refers to a rendition block given elsewhere?
           "start": "0",
           "end": "446",
           "timeunit": "http://cv.iptc.org/newscodes/timeunit/editUnit",
       },
       "language": "EN",
       "languagerole": "http://cv.iptc.org/newscodes/languagerole/voiceOver",
       "description": "Vicco von Bülow entering exhibition"
   }
]

Here's how it would look if icon was a rendition: (Note that we would have to change the schema to define a rendition type, and then refer to it from both "renditions" and "icon")

       "icon": {
           "mimetype": "image/jpeg",
           "width": 200,
           "height": 150,
           "sizeinbytes": 16748,
           "href": "http://www.example.com/video/Keyframes/407624.jpeg"
       },

bquinn commented 4 years ago

Questions:

Should we declare defaults for things like timeunit, to make things simpler?
The original request says we can include anything in ninjs "except for associations and renditions". But why would we want to ignore those? A part of a video might be associated with a URL, and a rendition could be used to represent a thumbnail. Which leads to the next question:
Should we just use renditions instead of icon?
If we want users to include valid ninjs properties in a part, then we have three choices:
1. To have some declared properties such as "partid", "sequencenumber" and "timedelimiter", and then a reference to a child object which can contain any valid ninjs property (what to call it?)
2. Or to put the new properties into the main ninjs schema as top-level properties, and just make "parts" a list of ninjs objects, the same way "associations" is defined.
3. We define in the schema the extra properties that are needed in part objects. The idea is that we shouldn't make things complicated for users just to make things easier to define in the schema. In this case it would be sensible to look at what fields are really needed, rather than just allowing any property to be used in a part. (BQ note to self: add examples of these three options to make it clear what we're talking about!)

bquinn commented 4 years ago

Use cases for parts:

Spatial parts (G2 construct is regionDelim):
- Image regions - defining people and products in an image
- Recommending crop boundaries for different devices / sizes
Time-based parts (G2 construct is timeDelim):
- describing segments of a radio broadcast
- declaring rights and copyright ownership of parts of a video
Would we need to declare metadata for a channel in a video stream? E.g. different channels of an MPEG file containing (possible place here for rights for the parts)

Notes:

We should allow overlapping parts - image regions will often overlap, and audio streams may overlap too
We could (should?) allow parts to be defined by both space and time - e.g. someone's face during an interview (as long as the person and the camera don't move!)

bquinn commented 4 years ago

Could subtitles / closed captions / audio description be a use case for parts in ninjs files?

For comparison, W3C has defined a timed text format, IMSC, based on TTML.

Here's an example. https://github.com/w3c/IMSC-1.1_Text_TestContent/blob/master/IMSC1-1_TEXT_Test-Reel_FMS_v4-0-1_2019-11-20.xml

bquinn commented 2 years ago

This issue came up again today when we discussed a requirement from TV2 Danmark.

We discussed W3C Media Fragments which are simple URI extensions that cover both temporal and spatial fragments: https://www.w3.org/TR/media-frags/

Examples of temporal fragments: http://ninsuna.elis.ugent.be/DownloadServlet/mfwg/fragf2f.ogv?t=21&track='ogg_1' http://ninsuna.elis.ugent.be/DownloadServlet/mfwg/fragf2f.ogv?t=npt:0:00:21.0 http://ninsuna.elis.ugent.be/DownloadServlet/mfwg/fragf2f.ogv?t=smpte-25:00:00:21:00&track='ogg_2'

Examples of spatial fragments:

xywh=160,120,320,240        # => results in a 320x240 box at x=160 and y=120
xywh=pixel:160,120,320,240  # => results in a 320x240 box at x=160 and y=120
xywh=percent:25,25,50,50    # => results in a 50%x50% box at x=25% and y=25%

jolla56 commented 1 year ago

"partmetas": [
   {
       "partid": "Part1_ID - should be a URI?",
       "icons": [{
           "href": "http://www.example.com/video/Keyframes/407624.jpeg" // could this be a rendition instead?
       }],
       "timedelimiter": [
{
           "renditionref": "http://example.com/vidrnd/avi25", // refers to a rendition block given elsewhere?
           "start": "0",
           "end": "446",
           "timeunit": "http://cv.iptc.org/newscodes/timeunit/editUnit",
       }, {
           "renditionref": "http://example.com/vidrnd/avi26", // refers to a rendition block given elsewhere?
           "start": "555",
           "end": "666",
           "timeunit": "http://cv.iptc.org/newscodes/timeunit/editUnit",

}
],
       "language": "EN",
       "languagerole": "http://cv.iptc.org/newscodes/languagerole/voiceOver",
       "description": "Vicco von Bülow entering exhibition"
   }
]

jolla56 commented 1 year ago

"partmetas": [
   {
       "partid": "Part1_ID - should be a URI?",
       "icons": [{
           "href": "http://www.example.com/video/Keyframes/407624.jpeg" // could this be a rendition instead?
       }],
       "regiondelimiters": [
         {
           "renditionref": "http://example.com/vidrnd/avi25", // refers to a rendition block given elsewhere?
           "x":  25,
            "y": 25,
           "width": 100,
            "height": 200
         }
       ],
       "language": "EN",
       "languagerole": "http://cv.iptc.org/newscodes/languagerole/voiceOver",
       "description": "Vicco von Bülow entering exhibition"
   }
]

bquinn commented 1 year ago

Questions raised today:

Which properties can be used inside a partMeta object (element of the partMeta array)? On one hand, we could allow anything in the ninjs schema, the same as we do in associations - but that's probably going too far. On the other, we could specify which properties can be re-used inside a partMeta and referring to their existing definition.
What would the link look like - the value of the renditionref property? @iyoung mentions JSONPath, it works well for them internally.

iyoung commented 1 year ago

Discussed naming, partMeta is consistently named for G2 along with itemMeta etc. Suggestion for NinJS would be 'parts'

iyoung commented 1 year ago

Discussed making parts similar to associations so we can include the whole NinJS structure, one particularly useful element would be rightsinfo.

iptc / newsinjson

Add support for parts CR 10-19 #19