Verify DCPs at sync speed on low power CPU laptops

[Cantar4]

The latest version is awesome ; with perfect color-correct images we played the Sarah mxf test-file at around 10fps! This is remarquable in a 2012 MacBook-Air 13". If QtAv can reach the 24fps speed necessary to verify the video/audio/subtitle sync of DCP films, this app will be all the rage in the filmmaking community ; not only in cinema schools and universities :)

To reach sync speed --without a power hungry GPU laptop-- I propose to transpose the process formerly used by cinematographers to save rawstock and make it appear more light sensitive, i.e., set the camera at 8fps, and show the film at 24fps by projecting each image three times. At Aaton we used to call it the [3x(1/3)] process. It seems to be unbelievable that eight samples per second is all you need to smoothly reproduce natural movements to the point the process escapes to some unwarned viewers (1) Its digital incarnation should be as simple as this sentence: pick one frame every three to the ffmeg engine (this reduces the CPU stress a lot), and once decoded send it three times to the ouput.

(1) for those who own a faster CPU (like the future 2016 Oct. MacBooks), a [2x(1/2)] option i.e. 12sps would be better even, I defy any one to see the 'subterfuge' at this sample rate. It's obvious but should be said, the subsampling doesn't apply to the audio nor subtitles mxf containers which are treated the standard way.

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/38008513-verify-dcps-at-sync-speed-on-low-power-cpu-laptops?utm_campaign=plugin&utm_content=tracker%2F307703&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F307703&utm_medium=issues&utm_source=github).

[wang-bin]

Do you mean decoding 1/3 number of frames, and rendering every decoded frame 3 times to get 30 fps display rate? The motion will not be smooth, is that acceptable?

[Cantar4]

yes, acceptable\ is the right world for a verification software on a 15" laptop screen. (btw, what is the j2k tile you use ; is it 500 or 1k pixel per line ?

**In 1969 at Eclair Corp of America we made an experiment in Los Angeles with non warned cinemagoers. We projected at 24fps, on a commercial cinema screen, three similar sequences shot in a Tango dance school respectively shot at 24, 12 and 8sps (using a special quartz motor I designed for the EclairNPR). Almost nobody could see the artifacts on the 12sps shots, whereas they felt something weird on the 8sps shots. We had to show them the Oude-Delft modified 16mm projector before a second passage to make them more scrutinizing... and amazed by the plasticity of the human brain ;)

[wang-bin]

And if recorded video is already at 8fps (1/3), decoded all frames and rendering at 24fps?

[wang-bin]

btw, what is the j2k tile you use ; is it 500 or 1k pixel per line ?

What is j2k tile? Where do i find it? All I know is the mxf you sent me is 1998x1080, i.e. 1998 pixels per line

[Cantar4]

[8fps video] Yes it would work, but this is not the real world, you must verify the very DCP three hours before the public projection. During a short-film festival in Berlin I even heard a story by which a flying doctor DCP monger was seated on a stool in the projection booth of a commercial cinema with his own fast GPU laptop running easyDCP.

[Cantar4]

[WB]: What is j2k tile? Where do i find it? All I know is the mxf you sent me is 1998x1080, i.e. 1998 pixels per line within the j2k encoded files you have four layers of tiles each half the resolution of the upper one ; this selection is done with the -lowres command line, that's why I was asking. If you kept the original's 2K res (which is unnecessary on a 15"), you probably are x4 slower than selecting 1K (a quarter of the weigth). Since QtAv is fast I guess you already selected 1k by any other means :).

[wang-bin]

What's the point of rendering at 24fps?

[Cantar4]

1- Because 24fps is now THE smpte standard (25, 30, & 2x fps are tolerated but not all projectors can handle thede rates). With a 24fps DCP in a NTFS or ex4 formatted USB key you can go to any cinema theater in the world and have your film projected in glorious XYZ space colors on a 15m wide screen. This is the beauty of DCP compared to the former 35mm film distribution or the pixilited mp4 for TV. The DC package is a must for students and independant filmmakers to be accepted by serious film festivals.

2- Because a DCP must be played at projection speed to verify if : the sound is not clipped (happens frequently with amateurish DCP creators) ; the timeline of the audio tracks has not been shifted for any cause ; the softsubtitles are properly 'framed' to the speech, and under the proper font and position ; there is no missing or 'green' passages.

[wang-bin]

Rendering at a specified frame rate is not hard. On my macbook air, the frame rate is about 5fps at the original resolution. If lowres=1, it's 11~12fps and looks smooth. If lowres=2, it can be 24fps without dropping any frame.

[Cantar4]

@wang this is very interesting, it shows that 2x(1/2 is probably a better compromise since by very nature it allows a better view on the possible video/audio sync issues @r-a-v-a-s since you have access to the -lowres settings in the ffmpeg guts, can you see on your retina screen the effects of the two lower resolutions (btw, how many pixel per line?) on the confort of viewing ?

[Cantar4]

Making of the Sarah demos : Take a 24fps short sequence with burned-in subtitles ; use a frame by frame player whith frame count insert (DCP-o-matic); take screenshots including the # of each image for later verification ; duplicate these screenshots ; import them as free picts in DoM ; add the sync-sound-track of the sequence ; add softsubtitles. Here is the DCP, later converted to a lighter mp4 for your examination.

Artefacts generated by the 2x(1/2) process (i.e. 12sps) are invisible but on Charlotte's head when she is quickly panning from left to right, her fingers on the cello show a life-like vivacity. The slowly descending arm of Sarah keeps its grace. The process is still usable at 3x(1/3) and 4x(1/4) for j2k verification ; much better than -lowres2 which produces soft and blurred images. I will send you the Sarah mp4s made from the DCPs.

[r-a-v-a-s]

@Cantar4 Sure... but I don't know where to add -lowres...

[wang-bin]

@r-a-v-a-s add av_dict_set(&d.dict, "lowres", "1", 0); before https://github.com/wang-bin/QtAV/blob/master/src/codec/AVDecoder.cpp#L110 The 3rd parameter can be 0, 1, 2, 3, 4,...

[r-a-v-a-s]

Thanks. There doesn't seem to be much difference when testing with the mxf from Cantar4's SarahOrigin_SHR-1_F-178_10_2K_20161003_SMPTE_OV package.

Something I noticed in testing is that any perceived lag seems to vanish when the timeline isn't visible, regardless of the lowres mode.

[wang-bin]

@r-a-v-a-s The resolution is different, and rendering fps is higher for lowres

[Cantar4]

I just installed your nightly build QtAV_Player-v1.11.0-91, and yes, It plays realtime jpeg2000 cinema images extracted from the Sarah DCP on my 1.7MHz MB-Air !!! Done in less than 3 weeks, this is a miracle considering the perfectly adequate temporal and spatial resolutions offered to our 13/15" laptops. Thank you for this achievement, QtAV is becoming the most ubiquitous AV player.

[r-a-v-a-s]

I just installed your nightly build QtAV_Player-v1.11.0-91

I sent him a build of the latest code with av_dict_set(&d.dict, "lowres", "1", 0); added.

[Cantar4]

After many tests we identified two options for QtAV as a DCP previewer:

"lowres2 and 3x[1/3]sps" will run at 24fps on a minimal laptop, i.e. CPU 1.7GHz, 4GB Ram. Displaying close to DVD resolution images, the lowres2 option will be ok for DCP short-film verification.

"lowres1 and 3x[1/3]sps" needs a slightly more gifted machine, i.e. CPU 2.9GHz, 8GB Ram. Lowres1 is displaying images as good as HD, this is more enjoyable during the long hours it takes to verify a feature-film DCP.

Hello Wang, is this todo list correct? -- play j2c_nnn.mxf convert XYZ to RGB, activate lowres1 image (done) user select lowres1 <-> lowres2 (todo)

-- play pcm_nnn.mxf QtAV already plays audio files (done)

-- display sub_nnn.mxf interpret mxf embeded XML subtitles (todo) QtAV already inserts soft-subtitles when playing MP4s.

-- automatically play the three mxf files together (todo)

--jp

[wang-bin]

Displaying real HD images What file type? I think this can be done without video player.

No problem to add lowres option in gui.

I need a video contains subtitle.

play the three files in sync On the same machine at the same time?

[Cantar4]

What file type? I think this can be done without video player.

as far as I know only your improved QtAV version in lowres-1 mode (HD equivalent) is able to play j2k files in realtime on a mid range laptop !!!

No problem to add lowres option in gui.

good news !

I need a video contains subtitle.

I will send you a complete DCP containing the three files together I tried to send you this package one month ago, I will give googledrive another chance !

(play the three files in sync) On the same machine at the same time?

Isn't what QtAV does when playing images, audio and subtitles from an MP4?

[wang-bin]

Isn't what QtAV does when playing images, audio and subtitles from an MP4?

Usually they are in a single file. Playing external audio track (from an external audio or video file) and external subtitle file is supported.

[Cantar4]

Is the Sarah package of any help to test this function ? If you confirm googledrive is not too cumbersome I will send you a light DCP in which each frame carries its burnt-in time code This timecode helps compare the performances of the CPU when QtAV converts '12sps into 24fps' versus '8sps into 24fps'.

[wang-bin]

Does the xml subtitle have a name? Where can I find it's spec? I use ffmpeg to decode subtitles, but ffmpeg can not recognize your xml subtitles

[Cantar4]

I will send you the coordinates of Carl Etherington of DCP-o-matic, he said he will be happy to help you on this matter.

[cth103]

There are two standards for DCP subtitles. The currently widely used one is known as "Interop"; spec is here:

http://www.deluxecdn.com/dcinema/support/ti_subtitle_spec_v1_1.pdf

The newer SMPTE standard is similar, but the specification is not freely available.