WebVMT: inclide reference to MISB standards

[ogcscotts]

Section 2.1 (Current Solutions) of the WebVMT draft (https://w3c.github.io/sdw/proposals/geotagging/webvmt/#currentsolutions) should reference the Motion Imagery Standards Board standards for encoding geospatial content along with video: http://www.gwg.nga.mil/misb/index.html.

[rjksmith]

Thanks for your comments.

My understanding of the MISB standards is that they embed key-length-values (KLVs) into an MPEG stream using SMPTE metadata registers, as described in the Editor's Draft Current Solutions, though I agree that the current wording is angled towards MXF as a specific example of this.

I'm happy to include further information about MISB standards if there are specific features which should be highlighted and that I've overlooked in the current analysis. I also refer you to the related issue #1020.

[ogcscotts]

@rjksmith Thanks for linking me to the original issue - I recall that it existed, but could not find it. I recommend a discussion in line with the comments on issue #1020 from @cmheazel. Many OGC members are involved in MISB or work with geotagged video, so there will be an expectation to assess MISB, at a minimum.

[cmheazel]

@rjksmith A motion imagery stream consists of multiple correlated data streams. You use the video and audio streams whenever you watch television. SMPTE has also defined a metadata stream. Since the streams are correlated through a common timestamp, the contents of the metadata stream will be correlated with the video, audio, and any other streams which may be included.

[rjksmith]

@ogcscotts I created a feature comparison table between WebVMT and other formats, including MPEG/MISB, based on the feedback from TPAC last year. Perhaps that would be worth adding to the Editor's Draft with some accompanying explanation, if it addresses your issue. Please advise.

Format/ Model	Location	Data Sync	Moving Objects	Video Sync	Video Cues	Map Controls
3GP	Yes	Camera Angle	One	Embed	No	No
KML	Yes	Yes	Multiple	No	Delete	Dynamic
Moving Features	Yes	Yes	Multiple	No	No	Static Bounds
MPEG/ MISB	Yes	Yes	One	Embed	No	No
Simple Features	Yes	No	No	No	No	No
WebVMT	Yes	Yes	Multiple	Link	Start & End	Dynamic

See below for amendment to MISB entry.

[cmheazel]

Now for the standards: SMPTE ST 336:2017 - Data Encoding Protocol Using Key-Length-Value MISB ST 0107.3 - KLV Metadata in Motion Imagery, Nov 2018 (baseline KLV requirements) MISB ST 0601.14 - UAS Datalink Local Set, Nov 2018 (core MISB metadata standard) MISB ST 0807.22 - MISB KLV Metadata Registry (extends SMPTE register)

There are many other MISB metadata standards. They are all extensions of MISB ST 0601 and recorded in MISB ST 0807. They include support for photogrammetry, tracking moving objects, sensor properties, meteorology, positional accuracy, and many more.

[cmheazel]

@rjksmith MISB 0903 supports multiple (thousands) of moving features. One issue we have faced is that you cannot display them all at one time.

[cmheazel]

Video Cues - MISB standards are built on SMPTE standards. They use the same time-stamp and frame count schemes as video and audio streams. So any tool which can insert video and audio cues should support cues on the metadata stream.

[cmheazel]

Map controls - MISB and SMPTE standards support streaming data. The map display (and associated map controls) are the responsibility of the client. As an aside, I believe that at one time (over ten years ago) we demonstrated generation of KML from the moving object detections in a MISB 0903 data stream. Biggest problem was that the KML rendering could not keep up with the data.

[cmheazel]

And the MISB standards are at this URL

[ogcscotts]

@rjksmith @cmheazel thanks to you both for getting more direct comparison information available.

[rjksmith]

@cmheazel Many thanks for the MISB references and link. I'll investigate further.

My feature comparison table (above) is only a draft and probably needs some refinement. However, it seems to have prompted a constructive discussion, and thanks for your feedback.

A key feature of WebVMT is that it's a linked file format, rather than embedded. There are pros and cons to both approaches, but it's particularly well-suited to web use and supports multiple media formats. I've also created an engagement site, since we last spoke (#1020), to provide a non-technical introduction to WebVMT at webvmt.org, which may offer more insight and includes a few tech demos.

Is there a user's guide/introduction to MISB standards somewhere - particularly with regard to location and moving objects? While technically correct, the standards documents are fairly indigestible and an overview or tutorial would be very helpful.

[lieberjosh]

Good (and overdue) discussion of georeferenced video models / formats. One question it would be good to address is suitability, e.g. what are different approaches best suited for.

MISB KLV standards are really aimed at high resolution, high frame rate video streams where a single (or sometimes dual) stream is the most efficient approach but requires very specific software support that is not widely available (although better than it used to be, e.g. GStreamer). It’s not great, but here is an FAQ — http://www.gwg.nga.mil/misb/faq.html http://www.gwg.nga.mil/misb/faq.html

WebVMT’s linked model utilizes existing web discovery and transport mechanisms to facilitate sharing of video and where it was shot. Much more interoperable across existing information sharing platforms, but may be less suitable to real time streaming and/or texture mapping video than MISB.

—Josh

On Apr 10, 2019, at 1:01 PM, Rob Smith notifications@github.com wrote:

@cmheazel https://github.com/cmheazel Many thanks for the MISB references and link. I'll investigate further.

My feature comparison table (above https://github.com/w3c/sdw/issues/1120#issuecomment-481740397) is only a draft and probably needs some refinement. However, it seems to have prompted a constructive discussion, and thanks for your feedback.

A key feature of WebVMT is that it's a linked file format, rather than embedded. There are pros and cons to both approaches, but it's particularly well-suited to web use and supports multiple media formats. I've also created an engagement site, since we last spoke (#1020 https://github.com/w3c/sdw/issues/1020), to provide a non-technical introduction to WebVMT at webvmt.org https://webvmt.org/, which may offer more insight and includes a few tech demos.

Is there a user's guide/introduction to MISB standards somewhere - particularly with regard to location and moving objects? While technically correct, the standards documents are fairly indigestible and an overview or tutorial would be very helpful.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/w3c/sdw/issues/1120#issuecomment-481775949, or mute the thread https://github.com/notifications/unsubscribe-auth/AExWhvgFKwH75g18wRYpsTrP4WSuj1aPks5vfhj6gaJpZM4ck6a-.

[cmheazel]

@lieberjosh MISB is best understood as suite of information transport standards. If WebVMT is positioned as a consumer of the transport stream, then I think we have a workable solution. Key to this will be to map from MISB KLV to WebVMT. After all, the information has to come from somewhere.

[rjksmith]

@lieberjosh Thanks for your feedback.

WebVMT is aimed at (and motivated by) the emerging markets in 'mobile video devices', e.g. drones, dashcams, body-worn video, and smartphones, and is designed as an enabler for sharing, presenting and indexing location (and other metadata) with video for the web. The current dashcam market provides a good example of the dichotomy between using proprietary formats to retain customers and allowing user content to be shared.

Thanks for the MISB FAQ link, which has answered some of my questions.

Real-time streaming is supported by WebVMT, so I'm interested to know why you think it may be less suitable for this application, and also what you mean by 'texture mapping video.'

[rjksmith]

@cmheazel Yes, exactly. WebVMT doesn't aim to replace existing solutions, such as MISB, but rather to provide a lightweight, accessible format for the web.

Input data can come from any sources that include video and location, with some way of aligning time correctly, and an MISB MPEG file certainly contains that information. Sketching out a mapping between MISB and WebVMT components sounds like an excellent idea.

I presume that the Motion Imagery Core Identifier (tag 94) allows MISB to discriminate between multiple moving objects, though I hadn't appreciated its significance before. Is there a way of labelling one object as the current video viewpoint in MISB?

Also, is there an MISB way of describing an area/volume of interest? For example, the land marked is used as a launch area by drone pilots and there's an exclusion zone around it for safety reasons.

[rjksmith]

I've updated the format comparison table (above) in light of the discussion.

Format/ Model	Location	Data Sync	Moving Objects	Video Sync	Video Cues	Map Controls
MPEG/ MISB	Yes	Yes	Multiple	Embed	No	Dynamic

Video cues are more than a timestamp and refer to W3C TextTrackCue and DataCue structures which have start and end times related to their content. For example, subtitle text can be displayed at a given time for a given duration with a single cue.

[rjksmith]

I've drafted a mapping between WebVMT commands and tags in MISB ST 0902.8 to highlight similarities and differences.

Map	lat	lng	rad	end/dur	path
pan	Center Lat [tag 23]	Center Lng [tag 24]	-	No	-
zoom	-	-	Target Width [tag 22]	-	-

Path	lat	lng	rad	end/dur	path
move	Sensor Lat [tag 13]	Sensor Lng [tag 14]	-	No	Core Id [tag 94]
line	Sensor Lat [tag 13]	Sensor Lng [tag 14]	-	No	Core Id [tag 94]

I've been unable to identify a way to mark an area in MISB, so have no mapping for WebVMT shape commands, though suggestions are welcome.

[rjksmith]

@cmheazel @lieberjosh @ogcscotts There's an agenda item to discuss 'video search with location' at the Spatial Data on the Web meeting on 25th June 2019 in Leuven, which is related to this issue and may be of interest to you.

I'd welcome your feedback in the meeting or advance comments via GitHub issue #1130, which outlines the discussion and its goals. Many thanks.

[ogcscotts]

@rjksmith I think that the scope of the proposed discussion looks clear with solid objectives. I will likely be unable to attend much of the SDWIG session due to my chair role in the overall TC Meeting, but I'll keep an eye on this repo.