mipops / dvrescue

Archivist-made software that supports data migration from DV tapes into digital files suitable for long-term preservation. Snapshot daily builds are at https://mediaarea.net/download/snapshots/binary/dvrescue/.
BSD 3-Clause "New" or "Revised" License
94 stars 20 forks source link

decklink capture has 2 video tracks #707

Closed dericed closed 5 months ago

dericed commented 1 year ago

I'm using the dvrescue found on the test computer.

% ./dvrescue/Project/GNU/CLI/dvrescue device://59:00000000:003bc300 -m out3.mkv

The output has two video tracks, no audio track, and an invalid EBML structure.

% ffmpeg -i out3.mkv  
ffmpeg version 6.0 Copyright (c) 2000-2023 the FFmpeg developers
  built with Apple clang version 12.0.0 (clang-1200.0.32.29)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/6.0 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libaribb24 --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
  libpostproc    57.  1.100 / 57.  1.100
[matroska,webm @ 0x7fb5c2e26240] Element at 0x1bf ending at 0x1cd exceeds containing master element ending at 0x1cc
[matroska,webm @ 0x7fb5c2e26240] Duplicate element
[matroska,webm @ 0x7fb5c2e26240] Element at 0x231 ending at 0xe7159 exceeds containing master element ending at 0x13f7
[matroska,webm @ 0x7fb5c2e26240] Duplicate element
[matroska,webm @ 0x7fb5c2e26240] 0x00 at pos 202 (0xca) invalid as first byte of an EBML number
[matroska,webm @ 0x7fb5c2e26240] Element at 0x67 ending at 0xd14dbb9253abf4 exceeds containing master element ending at 0x1413
[matroska,webm @ 0x7fb5c2e26240] 0x00 at pos 119 (0x77) invalid as first byte of an EBML number
[matroska,webm @ 0x7fb5c2e26240] Element at 0x1bf ending at 0x1cd exceeds containing master element ending at 0x1cc
[matroska,webm @ 0x7fb5c2e26240] Invalid track number 2
    Last message repeated 4 times
Input #0, matroska,webm, from 'out3.mkv':
  Metadata:
    encoder         : dvrescue0.00
  Duration: 00:00:12.38, start: 0.000000, bitrate: 226805 kb/s
  Stream #0:0: Video: v210 (v210 / 0x30313276), yuv422p10le(bottom first), 720x486, SAR 9:10 DAR 4:3, 29.97 fps, 29.97 tbr, 1k tbn (default)
  Stream #0:1: Video: v210 (v210 / 0x30313276), yuv422p10le(bottom first), 720x486, SAR 9:10 DAR 4:3, 29.97 fps, 29.97 tbr, 1k tbn (default)
At least one output file must be specified
dericed commented 1 year ago

Trying again on the latest daily build:

% dvrescue device://59:00000000:003bc300 --decklink-audio-source embedded --decklink-video-source sdi --decklink-video-mode ntsc -m test.mkv
Capture frame 132, press ctrl+c to stop.%                                                                                                       

% ffmpeg -i test.mkv
ffmpeg version 6.0 Copyright (c) 2000-2023 the FFmpeg developers
  built with Apple clang version 12.0.0 (clang-1200.0.32.29)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/6.0 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libaribb24 --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
  libpostproc    57.  1.100 / 57.  1.100
[matroska,webm @ 0x7fa185c04d80] Element at 0x1bf ending at 0x1cd exceeds containing master element ending at 0x1cc
[matroska,webm @ 0x7fa185c04d80] Duplicate element
[matroska,webm @ 0x7fa185c04d80] Element at 0x231 ending at 0xe7161 exceeds containing master element ending at 0x13f7
[matroska,webm @ 0x7fa185c04d80] Duplicate element
[matroska,webm @ 0x7fa185c04d80] 0x00 at pos 202 (0xca) invalid as first byte of an EBML number
[matroska,webm @ 0x7fa185c04d80] Element at 0x67 ending at 0xd14dbb9253abf4 exceeds containing master element ending at 0x1413
[matroska,webm @ 0x7fa185c04d80] 0x00 at pos 119 (0x77) invalid as first byte of an EBML number
[matroska,webm @ 0x7fa185c04d80] Element at 0x1bf ending at 0x1cd exceeds containing master element ending at 0x1cc
[matroska,webm @ 0x7fa185c04d80] Invalid track number 2
    Last message repeated 4 times
Input #0, matroska,webm, from 'test.mkv':
  Metadata:
    encoder         : dvrescue0.00
  Duration: 00:00:04.40, start: 0.000000, bitrate: 226806 kb/s
  Stream #0:0: Video: v210 (v210 / 0x30313276), yuv422p10le(bottom first), 720x486, SAR 9:10 DAR 4:3, 29.97 fps, 29.97 tbr, 1k tbn (default)
  Stream #0:1: Video: v210 (v210 / 0x30313276), yuv422p10le(bottom first), 720x486, SAR 9:10 DAR 4:3, 29.97 fps, 29.97 tbr, 1k tbn (default)
At least one output file must be specified

Still two video tracks and no audio. And invalid mkv structure.

% mediaconch test.mkv    
fail! test.mkv
 -- EBML-ELEMENT-CONTAINS-MANDATES
 --    [fail:TrackUID MUST be a Child Element of TrackEntry but is not present.]
 --     [/Segment[1]/Tracks[1]/TrackEntry[1]: [157 bytes] at bytes offset of 303]
 --    [fail:TrackUID MUST be a Child Element of TrackEntry but is not present.]
 --     [/Segment[1]/Tracks[1]/TrackEntry[2]: [70 bytes] at bytes offset of 460]

It seems like there are void elements where there are missing parts in the mkv tracks section so maybe it's the way FFmpeg is ending the capture. For instance the CRC and Track UID are not in the tracks element but there are void elements of those same sizes as if it should be overwritten at the end. Perhaps a different stop signal is needed so FFmpeg can finalize the file.

dericed commented 1 year ago

Trying this again with the 2023-1003-3 build.

$ dvrescue --list_devices
84:3c0a102a:00000000: UltraStudio HD Mini [DeckLink]
dvrescue device://84:3c0a102a:00000000 --decklink-video-source sdi --decklink-audio-source embedded --decklink-timecode-format vitc --decklink-video-mode ntsc -m test.mkv
Capture frame 9, press ctrl+c to stop.^

I captured an SDI stream for about 9 frames. Zipped and attached here:
test.mkv.zip

The file gives many errors:

from ffmpeg:

[matroska,webm @ 0x7f7ee9429dc0] Element at 0x1bf ending at 0x1cd exceeds containing master element ending at 0x1cc
[matroska,webm @ 0x7f7ee9429dc0] Duplicate element
[matroska,webm @ 0x7f7ee9429dc0] Element at 0x231 ending at 0xe7161 exceeds containing master element ending at 0x13f7
[matroska,webm @ 0x7f7ee9429dc0] Duplicate element
[matroska,webm @ 0x7f7ee9429dc0] 0x00 at pos 202 (0xca) invalid as first byte of an EBML number
[matroska,webm @ 0x7f7ee9429dc0] Element at 0x67 ending at 0xd14dbb9253abf4 exceeds containing master element ending at 0x1413
[matroska,webm @ 0x7f7ee9429dc0] 0x00 at pos 119 (0x77) invalid as first byte of an EBML number
[matroska,webm @ 0x7f7ee9429dc0] Element at 0x1bf ending at 0x1cd exceeds containing master element ending at 0x1cc
[matroska,webm @ 0x7f7ee9429dc0] Invalid track number 2

from mkvalidator

ERR200: Missing element 'TrackUID' in TrackEntry at 303
ERR200: Missing element 'TrackUID' in TrackEntry at 460
ERR200: Missing element 'TrackUID' in TrackEntry at 530
ERR068: The SeekPoint at 91 references an unknown Tags at 517
ERR067: The SeekPoint at 106 references a Cues at wrong position 8475773 (real 8475825)
WRN0B8: Track #3 is defined but has no frame
    file "test.mkv"
    created with dvrescue0.00 / dvrescue0.00

via mediaconch

fail! test.mkv
 -- MKV-ELEMENT-VALID-PARENT
 --    [fail:TrackEntry is not a valid Parent Element of BlockAdditionMapping.]
 --     [/Segment[1]/Tracks[1]/TrackEntry[1]/BlockAdditionMapping[1]: [13 bytes] at bytes offset of 447]
 --    [fail:BlockAdditionMapping is not a valid Parent Element of BlockAddIDType.]
 --     [/Segment[1]/Tracks[1]/TrackEntry[1]/BlockAdditionMapping[1]/BlockAddIDType[1]: 3224113 at bytes offset of 450]
 --    [fail:BlockAdditionMapping is not a valid Parent Element of BlockAddIDValue.]
 --     [/Segment[1]/Tracks[1]/TrackEntry[1]/BlockAdditionMapping[1]/BlockAddIDValue[1]: 121 at bytes offset of 456]
 -- EBML-ELEMENT-CONTAINS-MANDATES
 --    [fail:TrackUID MUST be a Child Element of TrackEntry but is not present.]
 --     [/Segment[1]/Tracks[1]/TrackEntry[1]: [157 bytes] at bytes offset of 303]
 --    [fail:TrackUID MUST be a Child Element of TrackEntry but is not present.]
 --     [/Segment[1]/Tracks[1]/TrackEntry[2]: [70 bytes] at bytes offset of 460]
 --    [fail:TrackUID MUST be a Child Element of TrackEntry but is not present.]
 --     [/Segment[1]/Tracks[1]/TrackEntry[3]: [31 bytes] at bytes offset of 530]
dericed commented 11 months ago

I'm testing with https://old.mediaarea.net/download/snapshots/binary/dvrescue/20231130-2/. I used /usr/local/bin/dvrescue -y device://84:3c0a102a:00000000 --control "/dev/tty.RS422 Deck Control" --decklink-video-mode ntsc --decklink-video-source composite --decklink-audio-source embedded --decklink-timecode-format vitc -capture -cmd play -m out35.mkv --verbosity 9 --csv

I'm attaching the output but it has many errors in ffmpeg.

$ ffmpeg -i out35.mkv 
ffmpeg version 6.0 Copyright (c) 2000-2023 the FFmpeg developers
  built with Apple clang version 13.0.0 (clang-1300.0.29.30)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/6.0_1 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libaribb24 --enable-libbluray --enable-libdav1d --enable-libjxl --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox --enable-audiotoolbox
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
  libpostproc    57.  1.100 / 57.  1.100
[matroska,webm @ 0x7f9b8c43a080] Element at 0x1bf ending at 0x1cd exceeds containing master element ending at 0x1cc
[matroska,webm @ 0x7f9b8c43a080] Duplicate element
[matroska,webm @ 0x7f9b8c43a080] Element at 0x231 ending at 0xe7159 exceeds containing master element ending at 0x13f7
[matroska,webm @ 0x7f9b8c43a080] Duplicate element
[matroska,webm @ 0x7f9b8c43a080] 0x00 at pos 202 (0xca) invalid as first byte of an EBML number
[matroska,webm @ 0x7f9b8c43a080] Element at 0x67 ending at 0xd14dbb9253abf4 exceeds containing master element ending at 0x1413
[matroska,webm @ 0x7f9b8c43a080] 0x00 at pos 119 (0x77) invalid as first byte of an EBML number
[matroska,webm @ 0x7f9b8c43a080] Element at 0x1bf ending at 0x1cd exceeds containing master element ending at 0x1cc
[matroska,webm @ 0x7f9b8c43a080] Invalid track number 2
    Last message repeated 4 times
Input #0, matroska,webm, from 'out35.mkv':
  Metadata:
    encoder         : dvrescue0.00
  Duration: 00:00:00.43, start: 0.000000, bitrate: 226816 kb/s
  Stream #0:0: Video: v210 (v210 / 0x30313276), yuv422p10le(bottom first), 720x486, SAR 9:10 DAR 4:3, 29.97 fps, 29.97 tbr, 1k tbn (default)
  Stream #0:1: Video: v210 (v210 / 0x30313276), yuv422p10le(bottom first), 720x486, SAR 9:10 DAR 4:3, 29.97 fps, 29.97 tbr, 1k tbn (default)
At least one output file must be specified

out35.mkv.zip

dericed commented 7 months ago

Testing with 24.03 release and this seems resolved:

$ dvrescue --version DVRescue v.24.03 (MediaInfoLib v.24.01) by MIPoPS.

drice:~ drice$ dvrescue --list_devices 2024-03-31 13:18:12.333 dvrescue[59738:1488982] WARNING: AVCaptureDeviceTypeExternal is deprecated for Continuity Cameras. Please use AVCaptureDeviceTypeContinuityCamera and add NSCameraUseContinuityCameraDeviceType to your Info.plist. 84:160a010c:00000000: UltraStudio HD Mini [DeckLink] drice:~ drice$ dvrescue device://84:160a010c:00000000 -m out4.mkv 2024-03-31 13:18:24.401 dvrescue[59755:1489247] WARNING: AVCaptureDeviceTypeExternal is deprecated for Continuity Cameras. Please use AVCaptureDeviceTypeContinuityCamera and add NSCameraUseContinuityCameraDeviceType to your Info.plist. Capture frame 29, press ctrl+c to stop.drice:~ drice$ drice:~ drice$ drice:~ drice$ ffmpeg -i out4.mkv ffmpeg version 6.1.1 Copyright (c) 2000-2023 the FFmpeg developers built with Apple clang version 15.0.0 (clang-1500.1.0.2.5) configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/6.1.1_3 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags='-Wl,-ld_classic' --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libaribb24 --enable-libbluray --enable-libdav1d --enable-libharfbuzz --enable-libjxl --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopenvino --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox --enable-audiotoolbox --enable-neon libavutil 58. 29.100 / 58. 29.100 libavcodec 60. 31.102 / 60. 31.102 libavformat 60. 16.100 / 60. 16.100 libavdevice 60. 3.100 / 60. 3.100 libavfilter 9. 12.100 / 9. 12.100 libswscale 7. 5.100 / 7. 5.100 libswresample 4. 12.100 / 4. 12.100 libpostproc 57. 3.100 / 57. 3.100 [matroska,webm @ 0x131905950] Unexpected BlockAdditions found in a Block from Track with TrackNumber 1 where MaxBlockAdditionID is 0 [aist#0:1/pcm_s32le @ 0x131907770] Guessed Channel Layout: stereo Input #0, matroska,webm, from 'out4.mkv': Metadata: encoder : DVRescue 24.03 Duration: 00:00:00.97, start: 0.000000, bitrate: 226821 kb/s Stream #0:0: Video: v210 (v210 / 0x30313276), yuv422p10le(bottom first), 720x486, SAR 9:10 DAR 4:3, 29.97 fps, 29.97 tbr, 1k tbn (default) Stream #0:1: Audio: pcm_s32le, 48000 Hz, 2 channels, s32, 3072 kb/s (default) Stream #0:2(eng): Subtitle: subrip (default) At least one output file must be specified


some mkvalidator errors:

$ mkvalidator out4.mkv ERR0B1: SimpleBlock at 946566 track #3 is not a keyframe ERR0B1: SimpleBlock at 1892583 track #3 is not a keyframe ERR0B1: SimpleBlock at 2838592 track #3 is not a keyframe ERR0B1: SimpleBlock at 3784609 track #3 is not a keyframe ERR0B1: SimpleBlock at 4730626 track #3 is not a keyframe ERR0B1: SimpleBlock at 5676635 track #3 is not a keyframe ERR0B1: SimpleBlock at 6622652 track #3 is not a keyframe ERR0B1: SimpleBlock at 7568661 track #3 is not a keyframe ERR0B1: SimpleBlock at 8514678 track #3 is not a keyframe ERR0B1: SimpleBlock at 9460695 track #3 is not a keyframe ERR0B1: SimpleBlock at 10406704 track #3 is not a keyframe ERR0B1: SimpleBlock at 11352721 track #3 is not a keyframe ERR0B1: SimpleBlock at 12298730 track #3 is not a keyframe ERR0B1: SimpleBlock at 13244747 track #3 is not a keyframe ERR0B1: SimpleBlock at 14190764 track #3 is not a keyframe ERR0B1: SimpleBlock at 15136773 track #3 is not a keyframe ERR0B1: SimpleBlock at 16082790 track #3 is not a keyframe ERR0B1: SimpleBlock at 17028799 track #3 is not a keyframe ERR0B1: SimpleBlock at 17974816 track #3 is not a keyframe ERR0B1: SimpleBlock at 18920833 track #3 is not a keyframe ERR0B1: SimpleBlock at 19866842 track #3 is not a keyframe ERR0B1: SimpleBlock at 20812859 track #3 is not a keyframe ERR0B1: SimpleBlock at 21758868 track #3 is not a keyframe ERR0B1: SimpleBlock at 22704885 track #3 is not a keyframe ERR0B1: SimpleBlock at 23650902 track #3 is not a keyframe ERR0B1: SimpleBlock at 24596911 track #3 is not a keyframe ERR0B1: SimpleBlock at 25542928 track #3 is not a keyframe ERR0B1: SimpleBlock at 26488937 track #3 is not a keyframe ERR0B1: SimpleBlock at 27434954 track #3 is not a keyframe .. file "out4.mkv" created with DVRescue 24.03 / DVRescue 24.03


and mediaconch errors:

$ mediaconch out4.mkv fail! out4.mkv -- MKV-ELEMENT-VALID-PARENT -- [fail:TrackEntry is not a valid Parent Element of BlockAdditionMapping.] -- [/Segment[1]/Tracks[1]/TrackEntry[1]/BlockAdditionMapping[1]: [13 bytes] at bytes offset of 455] -- [fail:BlockAdditionMapping is not a valid Parent Element of BlockAddIDType.] -- [/Segment[1]/Tracks[1]/TrackEntry[1]/BlockAdditionMapping[1]/BlockAddIDType[1]: 3224113 at bytes offset of 458] -- [fail:BlockAdditionMapping is not a valid Parent Element of BlockAddIDValue.] -- [/Segment[1]/Tracks[1]/TrackEntry[1]/BlockAdditionMapping[1]/BlockAddIDValue[1]: 121 at bytes offset of 464]



@jerome, these errors are expected with use of new BlockAdditionMapping?
JeromeMartinez commented 7 months ago

@jerome, these errors are expected with use of new BlockAdditionMapping?

Old MediaConch? Maybe I forgot somewhere to add the new MKV elements, I'll check.

dericed commented 7 months ago

hmm

$ /usr/local/bin/mediaconch --version
MediaConch Command Line Interface 23.10.20240323

$ /usr/local/bin/mediaconch out4.mkv
! out4.mkv

Expected? !

JeromeMartinez commented 7 months ago

This BlockAdditionMapping elements need to be added to MediaConch, please open an issue on MediaConch repo.