Playback/record of 608/708 CC and VANC

[hummelstrand]

This request comes from a LinkedIn discussion: All US broadcast stations, cable systems and satellite to home broadcasters -- even those that don't originate their own programming -- must have the ability to pass through any captions present in the programming they air. Most stations also have obligations to air at least some of their locally-produced programming with closed captions.

Equipment that doesn't at the very least pass CEA-708 captions (preferably in SMPTE-334 caption data packets) can't be considered by even the smallest broadcasters.

[TomKaltz]

The mix pipeline would have to be updated for ancillary data. There is also a problem to solve about which layer takes precedence on the output. For example when two producers with ancillary data are playing on different layers...how are these "mixed"?

On Tue, Apr 2, 2013 at 12:06 PM, Jonas Hummelstrand < notifications@github.com> wrote:

This request comes from a LinkedIn discussion: All US broadcast stations, cable systems and satellite to home broadcasters -- even those that don't originate their own programming -- must have the ability to pass through any captions present in the programming they air. Most stations also have obligations to air at least some of their locally-produced programming with closed captions.

Equipment that doesn't at the very least pass CEA-708 captions (preferably in SMPTE-334 caption data packets) can't be considered by even the smallest broadcasters.

— Reply to this email directly or view it on GitHubhttps://github.com/CasparCG/Server/issues/92 .

[hummelstrand]

I'm guessing a pass-through (from DeckLink producer input on to the Bluefish and DeckLink consumers) would be a good start.

[TomKaltz]

All data must pass through the mixer pipeline. Basically we'd have to have a command to designate which layer's ancillary data is passed through. Unless we come up with some sort of mixing strategy so that multiple layers can produce ancillary data for output to the consumers.

On Tue, Apr 2, 2013 at 12:29 PM, Jonas Hummelstrand < notifications@github.com> wrote:

I'm guessing a pass-through (from DeckLink producer input on to the Bluefish and DeckLink consumers) would be a good start.

— Reply to this email directly or view it on GitHubhttps://github.com/CasparCG/Server/issues/92#issuecomment-15786370 .

[kierank]

From an FFmpeg perspective you would need to modify the MPEG-2 and MPEG-4 decoders to output the caption packets along with the video data in a frame accurate fashion. Someone might be doing this already as part of improvements to VLC but I'm not 100% sure.

[maceyldn]

Along with this any chance we can embed program time code and/or station clock via VITC

[kierank]

You can get SD VITC from the Blackmagic but not HD VITC.

[TomKaltz]

@kierank Can BMD Decklink output 708 SDI embedded captioning?

[kierank]

It lets you set VANC data, so if you write the code to support 708, then yes.

[euphoricz]

This issue is much more complex than it sounds.

708 is stored in the vanc which requires a 10 bit pixel format.

Caspar currently passes 8 bit through the pipeline.

Converting an input to 8 bit strips out the vanc data, not to mention ffmpeg will also strip out any closed captions on encoding.

In order to use ffmpeg to encode a 10 bit pixel format such as YUV422P ffmpeg must be compiled to do so, based on this post on stackexchange it sounds like if ffmpeg can encode 10 bit it will no longer encode 8 bit: http://video.stackexchange.com/questions/13164/encoding-422-in-10-bit-with-libx264

FFmpeg seems to be the major stumbling block in this equation. I was recently in contact with a possible ffmpeg developer who is looking into 608 / 708 encoding in ffmpeg, may be best to let ffmpeg get sorted first. However I'm sure integrating a newly released version of ffmpeg which possibly has changes to the API is not of great interest either as far as CasparCG development is concerned.

https://forum.blackmagicdesign.com/viewtopic.php?f=12&t=38147

Not trying to dissuade because I would really like to see this implemented, only providing further information that I have come across in my own attempt to enable 608 / 708 closed caption pass through in Caspar (as well as CC from srt).

[euphoricz]

Another option could be just 608 pass through.

Many broadcasters will have a piece of hardware known as a closed caption legalizer which will encapsulate the 608 in 708 downstream.

If you can pull the 608 closed captions out of the video in the producer and pass them with the frame to the consumer thus bypassing any encoding / mixing, this theoretically should work.

Of course this doesn't take any layering / mixing into account.

[premultiply]

I think a good point to start would be an enhancement of Decklink consumer by adding a separate input pin for ancillary data. The Decklink consumer should continue to consume the same sample format as now but internally it should be mapped/converted to 10 bit and mixed with the ancillary data. This 10 bit stream will be passed to the Decklink driver.

Dont think of CC only. There are other popular subtitle formats like OP-47 or SMPTE2031. And of course other ANC data like AFD or Timecode may be even more important and perhaps easier to start with.

[TomKaltz]

Would love for this to go on current roadmap.

[ronag]

Not sure where funding for this would come from but I'll add it to 3.0 preliminarily and we'll see...

[premultiply]

Oha. Can be very complex. But ok, do you have an idea how this may look like on a multilayer CG system?

[TomKaltz]

I think we would have a mixer command that switches which layer to read captioning data from. It’s would probably be a good limitation to limit caption passing to one layer

-- Thomas Kaltz III 586-214-7150 Sent from Gmail Mobile

[premultiply]

Layer 0 :-)

But captioning/subtitling is not only this 6708 US stuff. „Rest of the world“ uses 436M-Tracks in MXF or separate EBU-STL-files to do OP-47 teletext subtitling.

[TomKaltz]

I could probably get some funding for this feature if there is anyone out there that can deliver. Please contact me tomkaltz gmail

[miguelmanticore]

Dear All,

Is someone working on feature? I'm available to help, is already the pipline using 10bits?

[TomKaltz]

This is not being worked on. I would love to find a way to get this implemented. The mixer pipeline is currently 8bit.

If there is anyone in the community that might be able to fund something like this please contact me: tomkaltz at gmail.

[miguelmanticore]

I will start a tour of the code and I will give you feedback.

Em 08/03/2018 17:08, "Thomas Kaltz III" notifications@github.com escreveu:

This is not being worked on. I would love to find a way to get this implemented. The mixer pipeline is currently 8bit.

If there is anyone in the community that might be able to fund something like this please contact me: tomkaltz at gmail.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/CasparCG/server/issues/92#issuecomment-371554151, or mute the thread https://github.com/notifications/unsubscribe-auth/AjdzgiY6zXIE-L0yg-26UMOrwvRdE_Dyks5tcWV3gaJpZM4Ai767 .

[premultiply]

Decklink „4k“ models provide an independant 10 bit interface for ANC-Data. So there is no need to capture and feed 10 bit image data in same format anymore.

[bobyorke]

I've noticed that the latest (10.11) SDK now has some new ancillary-data interfaces, and now allows for the pixel format of the ancillary data to differ from the active picture—and neither has to be 10-bit YUV! This looks to be quite a big step for handling ancillary data.

However, the documentation still states that for capture the pixel format for ancillary data has to match video capture (expect for the 4K models), so that's still potentially an issue. But there's no similar mention for the output, and the SDK samples include a CEA708 generator using an 8-bit (uint8_t) buffer for the ancillary data.

[carloshbranches]

Do Decklink and NDI cards currently allow recording / playing CC metadata? I would like to know, how is the situation of this issue today?

greetings, CHBM

[mint-dewit]

Do Decklink and NDI cards currently allow recording / playing CC metadata?

We use GitHub as a tracker for all features, bugs and public roadmap. As soon as there is any work done on this feature it will be reported on this issue, thus you can conclude no work has been done on this.

[tvadi]

Seeing if there is any update here. CasparCG cannot pass captions? Thanks!

[tvadi]

Hi, would anyone have any guidance on getting captions to pass though Decklink in CasparCG? Using 4k extreme and cannot get my files with embedded captions to show captions, the files car mepg-2 with both 608 and 708. Willing to pay a developer.. Thanks!

[carloshbranches]

hello @tvadi I would also like to know, but lately I have been researching this subject. Amazingly, I found something about it, but I just wanted to know if someone compiled the developed version of Casparcg, found at this link: in2ip/server The developers people use, if I'm not mistaken, the casparcg 2.1.0 version, I wish I had been able to compile, even with the instructions given, but I couldn't compile.

greetings,

[tvadi]

@carloshbranches ..I emailed @gizahNL about that, he had been working on it but is no longer, his reply is here: "CEA608/708 in the end never materialized into a requirement so we didn't explore or test further. Since our usecase is quite limited (ffmpeg in only, ndi out only, Linux only) I only coded support for reading the CC stream via the ffmpeg module and outputting in a custom format to NDI. The only issue we saw was with framerate conversion done by the CasparCG mixer..."

So seems the framerate conversion would be an issue, totally get that but for me, we would have files all the same frame rate and same resolution, so it would not have to do any conversion. I don't even need any graphic overlays or anything really, just to play a 24/7 schedule of all the same file types. ..Or even if CasparCG could accept a UPD/TCP stream input from VLC or something and just pass the captions on out the Decklink SDI, that would also be amazing.. In my case I just need a play out server, so not really interested in SDI input captions, if anyone is working on this, I think just getting output would be the best place to start.

I have done a lot of research on this and one of the main things is that Blackmagic Design really sucks at support when it comes to passing captions out the Decklinks.. I have posted on every page on their forum on the topic on - nothing.. I have been writing weekly to the developer support, their only real solution is to dig into the SDK, they only have one good functioning sample in the SDK and it does not even play a file, just puts out a message on SDI saying "CEA-708 Closed Captions Second line of text!" That sample can be found here in the SDK: Win\Samples\bin\ClosedCaptions.exe

So good news is it will pass captions, they just do not make it easy.. The closest thing I can find that plays files with embedded captions out the Decklink board and still passes closed captions is from the Medialooks\MFormats SDK\Samples\C#\Sample File Playback\bin\x64\Debug> Sample File Playback.exe

gizahNL could help possibly for a fee, @carloshbranches maybe if you are interested we could raise some funds for this development. Guessing this is not huge on anyone's to do lists, just testing/ viewing 608/708 captions requires expensive SDI broadcast monitors, I am borrowing one from work.. I have a test server and decklink cards, more than willing to let someone VPN into the unit and try out some builds, I would be able to verify captions on the fancy monitor. Hope all are well and healthy, thanks to all working on CasparCG- Cheers- Matt

[tvadi]

Here are some short test files with embedded captions if anyone wants to try digging into this...

https://www.dropbox.com/s/b0lufj69tyitcbn/1080iSampleFileCC.mpg?dl=0 https://www.dropbox.com/s/ujyna1dx3bhs7p4/720ts_6DK01H_tru720og.ts?dl=0

Could do MP4 or MXF with embedded captions if that is easier- https://www.dropbox.com/s/8jrlypcb38t9ggn/1080mp4cc8826881M544356H.mp4?dl=0 https://www.dropbox.com/s/jrhzd7kkjfh7kyb/1080mxf_708_I28706.mxf?dl=0

The .TS extension can be manually changed to .mpg without issue. You can view the captions in VLC player (subtitle>sub track>closed captions 1) and with MediaInfo, which will list: Text #1 ID : 4096 (0x1000)-CC1 Menu ID : 1 (0x1) Format : EIA-608 Muxing mode : A/53 / DTVCC Transport Muxing mode, more info : Muxed in Video #1 Duration : 28mn 29s Bit rate mode : Constant Stream size : 0.00 Byte (0%) Language : English

Text #2 ID : 4096 (0x1000)-1 Menu ID : 1 (0x1) Format : EIA-708 Muxing mode : A/53 / DTVCC Transport Muxing mode, more info : Muxed in Video #1 Duration : 28mn 29s Bit rate mode : Constant Stream size : 0.00 Byte (0%) Language : English

[dedicatedbroadcastsolutions]

I would really like to see some progress on this project. I have seen several people offer to fund this project on the forums but it is unclear if they have been connected with the correct people to make this happen. Here are some things I have observed when fixing captions at a local TV station in the USA.

Most commercial playout solutions require specific input file formats for captions support and specifically do not support frame rate conversion while processing captions. (that would require a software captions encoder/transcoder to change framerate)

While 708 captions support would be nice, it appears to me that most broadcast television in the USA is actually 608 or 608 wrapped in 708 instead of containing true 608 and 708 streams. This is because many signals need to support down conversion either for distribution or for old SD ATSC 1.0 set top boxes so 608 has to be present and valid. Most seem to thus focus on a functional 608 pipeline which is wrapped in 708 for HD and neglect having a full 708 pipeline along with a valid 608 stream as that would require duplication of work when creating the captions due to the differing feature set. A legal 708 pipeline must contain and preserve the 608 wrapped within so most just leave it at that.

In my case I would be interested in a pipeline where files of the correct format and frame rate are played by a file consumer and the captions layer is routed to a deck link consumer. Any frame rate conversion should be done outboard in hardware that supports caption frame rate conversion until that feature can be integrated in software at a later time. Our current legacy library is at 1080i5994 mxf with SMPTE 436M captions stream. Broadcasting for ATSC 3.0 and ATSC 1.0 datacasting though is more likely to be 30p or 60p so matching everything to the internal progressive frame rate of casparCG seems reasonable. On the other hand as long as the source file/consumer frame rate matches the decklink output frame rate then captions should be able to be routed correctly without corruption even if the image gets converted for processing.