add format and stream metadata

[dericed]

QCTools will soon be able to push <streams> and <format> data to signalserver.

Here is an example:

<?xml version="1.0" encoding="UTF-8"?>
<!-- Created by QCTools 0.6.0 -->
<ffprobe:ffprobe xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' xmlns:ffprobe='http://www.ffmpeg.org/schema/ffprobe' xsi:schemaLocation='http://www.ffmpeg.org/schema/ffprobe ffprobe.xsd'>
    <program_version version="git-2017-02-05-e57fd92" copyright="Copyright (c) 2007-2017 the FFmpeg developers" build_date="Feb  8 2017" build_time="20:52:25" compiler_ident="not available" configuration="not available"/>
    <library_versions>
        <library_version name="libavutil" major="55" minor="46" micro="100" version="3616356" ident="Lavu55.46.100"/>
        <library_version name="libavcodec" major="57" minor="75" micro="100" version="3754852" ident="Lavc57.75.100"/>
        <library_version name="libavformat" major="57" minor="66" micro="101" version="3752549" ident="Lavf57.66.101"/>
        <library_version name="libavfilter" major="6" minor="73" micro="100" version="412004" ident="Lavfi6.73.100"/>
        <library_version name="libswscale" major="4" minor="3" micro="101" version="263013" ident="SwS4.3.101"/>
    </library_versions>
    <frames>
        <frame media_type="video" stream_index="0" key_frame="1" pkt_pts="0" pkt_pts_time="0.0000000" pkt_duration_time="0.0333667" pkt_pos="36" pkt_size="337252" width="720" height="486" pix_fmt="yuv422p10le" pict_type="I">
            <tag key="lavfi.signalstats.YMIN" value="4"/>
            <tag key="lavfi.signalstats.YLOW" value="119"/>
            <tag key="lavfi.signalstats.YAVG" value="235.387"/>
            <tag key="lavfi.signalstats.YHIGH" value="460"/>
            <tag key="lavfi.signalstats.YMAX" value="861"/>
            <tag key="lavfi.signalstats.UMIN" value="368"/>
            <tag key="lavfi.signalstats.ULOW" value="471"/>
            <tag key="lavfi.signalstats.UAVG" value="500.972"/>
            <tag key="lavfi.signalstats.UHIGH" value="515"/>
            <tag key="lavfi.signalstats.UMAX" value="578"/>
            <tag key="lavfi.signalstats.VMIN" value="356"/>
            <tag key="lavfi.signalstats.VLOW" value="415"/>
            <tag key="lavfi.signalstats.VAVG" value="485.547"/>
            <tag key="lavfi.signalstats.VHIGH" value="513"/>
            <tag key="lavfi.signalstats.VMAX" value="530"/>
            <tag key="lavfi.signalstats.VDIF" value="0"/>
            <tag key="lavfi.signalstats.UDIF" value="0"/>
            <tag key="lavfi.signalstats.YDIF" value="0"/>
            <tag key="lavfi.signalstats.SATMIN" value="0"/>
            <tag key="lavfi.signalstats.SATLOW" value="2"/>
            <tag key="lavfi.signalstats.SATAVG" value="30.3061"/>
            <tag key="lavfi.signalstats.SATHIGH" value="108"/>
            <tag key="lavfi.signalstats.SATMAX" value="182"/>
            <tag key="lavfi.signalstats.HUEMED" value="44"/>
            <tag key="lavfi.signalstats.HUEAVG" value="136.591"/>
            <tag key="lavfi.signalstats.TOUT" value="0.00519547"/>
            <tag key="lavfi.signalstats.VREP" value="0"/>
            <tag key="lavfi.signalstats.BRNG" value="0.0266918"/>
            <tag key="lavfi.cropdetect.x1" value="0"/>
            <tag key="lavfi.cropdetect.x2" value="720"/>
            <tag key="lavfi.cropdetect.y1" value="0"/>
            <tag key="lavfi.cropdetect.y2" value="486"/>
            <tag key="lavfi.cropdetect.w" value="720"/>
            <tag key="lavfi.cropdetect.h" value="486"/>
            <tag key="lavfi.psnr.mse.v" value="0"/>
            <tag key="lavfi.psnr.mse.u" value="0"/>
            <tag key="lavfi.psnr.mse.y" value="0"/>
            <tag key="lavfi.psnr.psnr.v" value="0"/>
            <tag key="lavfi.psnr.psnr.u" value="0"/>
            <tag key="lavfi.psnr.psnr.y" value="0"/>
            <tag key="lavfi.ssim.All" value="0"/>
            <tag key="lavfi.ssim.V" value="0"/>
            <tag key="lavfi.ssim.U" value="0"/>
            <tag key="lavfi.ssim.Y" value="0"/>
            <tag key="lavfi.idet.single.bff" value="0"/>
            <tag key="lavfi.idet.single.tff" value="0"/>
            <tag key="lavfi.idet.single.progressive" value="0"/>
            <tag key="lavfi.idet.single.undetermined" value="0"/>
            <tag key="lavfi.idet.multiple.bff" value="0"/>
            <tag key="lavfi.idet.multiple.tff" value="0"/>
            <tag key="lavfi.idet.multiple.progressive" value="0"/>
            <tag key="lavfi.idet.multiple.undetermined" value="0"/>
            <tag key="lavfi.idet.repeated.bottom" value="0"/>
            <tag key="lavfi.idet.repeated.top" value="0"/>
            <tag key="lavfi.idet.repeated.neither" value="0"/>
            <tag key="pkt_duration_time" value="0.0333667"/>
            <tag key="pkt_size" value="337252"/>
        </frame>
        <frame media_type="video" stream_index="0" key_frame="0" pkt_pts="33.3667" pkt_pts_time="0.0333667" pkt_duration_time="0.0333667" pkt_pos="349576" pkt_size="336151" width="720" height="486" pix_fmt="yuv422p10le" pict_type="I">
            <tag key="lavfi.signalstats.YMIN" value="4"/>
            <tag key="lavfi.signalstats.YLOW" value="119"/>
            <tag key="lavfi.signalstats.YAVG" value="235.289"/>
            <tag key="lavfi.signalstats.YHIGH" value="459"/>
            <tag key="lavfi.signalstats.YMAX" value="874"/>
            <tag key="lavfi.signalstats.UMIN" value="376"/>
            <tag key="lavfi.signalstats.ULOW" value="471"/>
            <tag key="lavfi.signalstats.UAVG" value="500.745"/>
            <tag key="lavfi.signalstats.UHIGH" value="515"/>
            <tag key="lavfi.signalstats.UMAX" value="579"/>
            <tag key="lavfi.signalstats.VMIN" value="358"/>
            <tag key="lavfi.signalstats.VLOW" value="416"/>
            <tag key="lavfi.signalstats.VAVG" value="485.555"/>
            <tag key="lavfi.signalstats.VHIGH" value="513"/>
            <tag key="lavfi.signalstats.VMAX" value="539"/>
            <tag key="lavfi.signalstats.VDIF" value="2.93949"/>
            <tag key="lavfi.signalstats.UDIF" value="3.89837"/>
            <tag key="lavfi.signalstats.YDIF" value="7.6502"/>
            <tag key="lavfi.signalstats.SATMIN" value="0"/>
            <tag key="lavfi.signalstats.SATLOW" value="2"/>
            <tag key="lavfi.signalstats.SATAVG" value="30.3544"/>
            <tag key="lavfi.signalstats.SATHIGH" value="108"/>
            <tag key="lavfi.signalstats.SATMAX" value="175"/>
            <tag key="lavfi.signalstats.HUEMED" value="44"/>
            <tag key="lavfi.signalstats.HUEAVG" value="132.945"/>
            <tag key="lavfi.signalstats.TOUT" value="0.0051126"/>
            <tag key="lavfi.signalstats.VREP" value="0"/>
            <tag key="lavfi.signalstats.BRNG" value="0.0265947"/>
            <tag key="lavfi.cropdetect.x1" value="0"/>
            <tag key="lavfi.cropdetect.x2" value="720"/>
            <tag key="lavfi.cropdetect.y1" value="0"/>
            <tag key="lavfi.cropdetect.y2" value="486"/>
            <tag key="lavfi.cropdetect.w" value="720"/>
            <tag key="lavfi.cropdetect.h" value="486"/>
            <tag key="lavfi.psnr.mse.v" value="0"/>
            <tag key="lavfi.psnr.mse.u" value="0"/>
            <tag key="lavfi.psnr.mse.y" value="0"/>
            <tag key="lavfi.psnr.psnr.v" value="0"/>
            <tag key="lavfi.psnr.psnr.u" value="0"/>
            <tag key="lavfi.psnr.psnr.y" value="0"/>
            <tag key="lavfi.ssim.All" value="0"/>
            <tag key="lavfi.ssim.V" value="0"/>
            <tag key="lavfi.ssim.U" value="0"/>
            <tag key="lavfi.ssim.Y" value="0"/>
            <tag key="lavfi.idet.single.bff" value="0"/>
            <tag key="lavfi.idet.single.tff" value="0"/>
            <tag key="lavfi.idet.single.progressive" value="0"/>
            <tag key="lavfi.idet.single.undetermined" value="0"/>
            <tag key="lavfi.idet.multiple.bff" value="0"/>
            <tag key="lavfi.idet.multiple.tff" value="0"/>
            <tag key="lavfi.idet.multiple.progressive" value="0"/>
            <tag key="lavfi.idet.multiple.undetermined" value="0"/>
            <tag key="lavfi.idet.repeated.bottom" value="0"/>
            <tag key="lavfi.idet.repeated.top" value="0"/>
            <tag key="lavfi.idet.repeated.neither" value="0"/>
            <tag key="pkt_duration_time" value="0.0333667"/>
            <tag key="pkt_size" value="336151"/>
        </frame>
    </frames>
    <streams>
        <stream index="0" codec_name="ffv1" codec_long_name="FFmpeg video codec #1" codec_type="video" codec_time_base="100/2997" codec_tag_string="FFV1" codec_tag="0x31564646" width="720" height="486" coded_width="720" coded_height="486" has_b_frames="0" sample_aspect_ratio="10:11" display_aspect_ratio="400:297" pix_fmt="yuv422p10le" level="-99" field_order="bt" refs="1" r_frame_rate="2997/100" avg_frame_rate="2997/100" time_base="1/11988" start_pts="0" start_time="0.000000">
            <disposition default="1" dub="0" original="0" comment="0" lyrics="0" karaoke="0" forced="0" hearing_impaired="0" visual_impaired="0" clean_effects="0" attached_pic="0" timed_thumbnails="0"/>
            <tag key="language" value="eng"/>
            <tag key="handler_name" value="DataHandler"/>
            <tag key="encoder" value="ffv1"/>
        </stream>
        <stream index="1" codec_name="pcm_s24le" codec_long_name="PCM signed 24-bit little-endian" codec_type="audio" codec_time_base="1/48000" codec_tag_string="in24" codec_tag="0x34326e69" sample_fmt="s32" sample_rate="48000" channels="2" channel_layout="stereo" bits_per_sample="24" r_frame_rate="0/0" avg_frame_rate="0/0" time_base="1/48000" start_pts="0" start_time="0.000000" bits_per_raw_sample="24">
            <disposition default="1" dub="0" original="0" comment="0" lyrics="0" karaoke="0" forced="0" hearing_impaired="0" visual_impaired="0" clean_effects="0" attached_pic="0" timed_thumbnails="0"/>
            <tag key="language" value="eng"/>
            <tag key="handler_name" value="DataHandler"/>
        </stream>
    </streams>
    <format filename="/Users/davidrice/Desktop/Dave/qctools_dhc_samples/ffv1/DHC0386_Ghosting.mov_ffv1.mov" nb_streams="2" nb_programs="0" format_name="mov,mp4,m4a,3gp,3g2,mj2" format_long_name="QuickTime / MOV" start_time="0.000000" duration="10.678000" size="111637099" bit_rate="83638957" probe_score="100">
        <tag key="major_brand" value="qt  "/>
        <tag key="minor_version" value="512"/>
        <tag key="compatible_brands" value="qt  "/>
        <tag key="encoder" value="Lavf55.13.102"/>
    </format>
</ffprobe:ffprobe>

This provides much more info about the file as a whole than we had before (where it had been mostly about frames).

I think we need a design of what data to use and how to present it and then to integrate it into a file view. I suggest these values are relevant to include in display:

• stream/@codec_name
• stream/@codec_type
• stream/@width (video)
• stream/@height (video)
• stream/@sample_aspect_ratio (video)
• stream/@display_aspect_ratio (video)
• stream/@pix_fmt (video)
• stream/@field_order (video)
• stream/@avg_frame_rate (video)
• stream/@sample_rate (audio)
• stream/@bits_per_raw_sample (audio)
• stream/@channels (audio)
• format/@filename
• format/@format_log_name
• format/@duration
• format/@size
• format/@bit_rate

[yayoiukai]

looks good. Please send me the sample file when it is ready.

[dericed]

also suggesting that along with stream and format data we list the filters used in the first frame

[dericed]

So how's this:

• @ablwr does mockup of a per file page listing stream/format data (the qcli utility can easily make data samples to show what sorts of stream and format data is available). Then link to or show mockups here for feedback.
• suggestions from @ablwr on navigation to/from the per-file page
• @yayoiukai integrates

More details needed?

[ablwr]

Will do!

[ablwr]

When you say the per-file page, do you mean this? http://192.168.99.100:8000/signals/get_graph/

Right now you can navigate to it when you go to File Process Status. Would be good to link to each individual file's page when they are listed in the graph sections though (or anywhere they are linked). This happens dynamically so I didn't boldly try to add the link myself.

[dericed]

Not sure. I think that page shows the result of a policy applied to a file. I was referring to a page that focused on the file itself.

[ablwr]

But re: mockups, I don't think we should do anything more complex than listing them, at least not at this time. Like this? I pushed a commit with the changes (should auto-link above).

Bonus: reporting is back!!!

[ablwr]

Ohhhh. I would also like to see and know how to get to that page, I'm not sure how to navigate (or create navigation towards) file 'show' pages. @yayoiukai?

[ablwr]

bump

[yayoiukai]

@ablwr I deleted file show page a while ago. I could add it back if there is a need for it. All process becomes either file or group base and depends on policy. So it is separated now. (And sorry it took me so long to get back to you.)

[yayoiukai]

also, it will be wonderful you can send me files. Thank you @dericed

[dericed]

I sent a few dozen files with stream and format sections.

[yayoiukai]

Thank you, I got them. I will bring file show page back and show all these metadata.

[yayoiukai]

@dericed I can't find some of the fields you mentioned above on the file you sent me or you posted above. What should I do? (For example, there is no mention of the size so I will only know QLtool files. )

The field missings are:

size, bitrate, sample_rate, bits_per_raw_sample, channel, duration.

I just do without them but please let me know!

[yayoiukai]

So... for this metadata, I decided to read it when the file is uploaded. I also brought back show file page too. Also moving on, all QCtool files will have this stream or depend on the file?