Metadata for video frame number streams?

[chkothe]

Hey, is there existing practice for how streams holding video frame numbers may be annotated in terms of stream type and metadata? (I know some folks have been writing such integrations quite a while ago, e.g. @mgrivich). Also cc @cboulay (I saw that blackrock integration).

Off the top of my head, I think if nothing else the relative path to the actual video file should be written into the stream metadata (if the video is in a known relative location to the XDF), or otherwise at least the filename without any path information. I'd keep absolute paths out of the spec for obvious reasons.

In terms of stream type, one may be tempted to use the VideoCompressed type and allow in lieu of the <encoding> tag a different tag that indicates that the video is external to the XDF (e.g., <external>). Or we might invent a new tag like VideoExternal, which may avoid a failure mode in XDF/LSL pplications that expect VideoCompressed to contain the actual compressed video but then just find a frame stamp in it (after all, that sort of disambiguation is what stream types are therefore).

[cboulay]

I have no strong preference. If I had to pick, I think I prefer VideoExternal over VideoCompressed.

Other lesser options:

• FrameCounts
• FrameIndices

[mgrivich]

This is what my old VideoStream app uses:

lsl_streaminfo info = lsl_create_streaminfo("VideoStream_0","VideoStream_0",1,requestedFrameRate,cft_int32,"");
lsl_xml_ptr desc = lsl_get_desc(info);
lsl_xml_ptr chn = lsl_append_child(desc, "channels");
lsl_append_child_value(chn, "name","frame");
lsl_append_child_value(chn,"unit","number");

lsl_xml_ptr sync = lsl_append_child(desc, "synchronization");
lsl_append_child_value(sync, "can_drop_samples", "true");

I don't see any need for you to keep to this. I'm not sure that anyone uses that app anymore.