Open JMY1000 opened 2 years ago
@JMY1000 just to clarify, when you say your deck has a TBC, do you mean it has one built in or it is hooked up to an external TBC or DPS and then routed to your video card? Also, how is your audio hooked up to your video card/routed to your computer?
@JMY1000 just to clarify, when you say your deck has a TBC, do you mean it has one built in or it is hooked up to an external TBC or DPS and then routed to your video card? Also, how is your audio hooked up to your video card/routed to your computer?
TBC is built into the deck[s]. Audio is coming directly out of the deck and going right into the card.
@JMY1000 it's been my experience that with most decks (even ones that are really fancy professional models) the built-in TBC sort of starts to break down and even if it's in decent shape, doesn't seem to be able to keep up with dropped frames and buffer overruns as well as a separate TBC/DPS unit will, especially when it comes to Hi8 and VHS. This will happen during playback as well if you are using the same set-up to view the videotapes. From your notes about having successful captures without dropped frames when using a DV input for Digital8, I would suspect the built-in TBC is the root cause since this wouldn't be utilized via Firewire since it's transferring the audio and video as digital vs. analog.
This will happen during playback as well if you are using the same set-up to view the videotapes.
Not sure if I'm reading this right, but the audio sync issues don't seem to crop up on the deck during normal playback, only on the capture computer. There also aren't any sync issues when recording with VirtualDub2 instead of vrecord, which makes me suspect it's not the TBC—especially since the results are the same with both decks. I would test with an external TBC just to confirm this, but I unfortunately don't have one and they're quite pricy. I'd be happy to drive over or send a tape if you'd be willing to test it though!
I apologize, I meant that the dropped frames will be present during playback. In my experience, when a TBC/DPS cannot keep up with dropped frames or buffering other issues, it results in PTS discontinuities and buffer overruns that then cause the audio to be out of sync. It might also have to do with the difference between your Windows set-up and your Mac because given the age of the Mac and the length of the videos, that might be a factor as well, but I'm not familiar with VirtualDub2's features/functions.
Ah, gotcha, no worries.
In my experience, when a TBC/DPS cannot keep up with dropped frames or buffering other issues, it results in PTS discontinuities and buffer overruns that then cause the audio to be out of sync
The logs don’t show any evidence of buffer overruns, only periods where no input is detected. I’m not sure if there’s a significant distinction on the vrecord side between the two though.
It might also have to do with the difference between your Windows set-up and your Mac because given the age of the Mac and the length of the videos
I don’t believe it’s related to the capture computers, as CPU load is quite low during capture (even on the Mac Pro) and Blackmagic Media Express also drops frames when running on Windows.
I’m definitely not beyond ruling out the TBC entirely, but since I don’t have an external one, I unfortunately can’t really pursue that route (at least not without sending tapes to someone else—something I’d be happy to try.) Ultimately though, I just need a way to capture the parts of the tape that do have video, not the parts that aren’t intact; the best TBC in the world can’t make up video that isn’t there. VirtualDub2 and the in-camera digitizer prove that can be done on the capture side; I’d like to make it that vrecord can do it to. If that means there needs to be some development work, I can definitely try and put in; I just don’t know if that needs to happen here in vrecord, upstream in ffmpeg, or somewhere else.
Just wanted to check in and see if there are any updates on this.
Hi @JMY1000, no updates that I am aware of - in my experience on Mac and Linux I've never been able to get a decent capture using a Blackmagic card without an external TBC. Since vrecord is dependent upon Blackmagic hardware, I don't see that being particularly fixable from the vrecord side.
Not sure why you would be seeing a different outcome on Windows with VirtualDub2 though! I unfortunately don't have the capability of testing that out on my end right now due to my hardware.
@privatezero Hmm, okay, thanks. What's the reason for external TBCs being more robust than internal TBCs? Also, what TBC[s] are you using for handling damaged media?
Hi @JMY1000 - can't say I know exactly why the externals seem to do better when used in this process. In terms of what kind we use, for the most part we use Leitch DPS-475s - believe @libbyhopfauf uses the same.
Hi @JMY1000 - can't say I know exactly why the externals seem to do better when used in this process. In terms of what kind we use, for the most part we use Leitch DPS-475s - believe @libbyhopfauf uses the same.
MIPoPS uses the Leitch DPS-575s (which are from the same line). I don't know all the engineering specifics either, but my basic understanding is that the external DPS is able to more consistently maintain a clean, stable signal from the deck to the computer, as well as identify/correct tape dropouts that are present on the tape (from breakdown/loss of magnetic coating). The professional, external ones are more consistently able to do this because they are more robust and offer more control over the various levels and signals. For the most part, they are also newer than the built-in ones, so they've experienced less wear-and-tear. I'm sure there's more to it than that, but those are the main reasons they are recommended/we use them.
As @privatezero mentioned though, I'm not sure why this isn't happening on your Windows set up and we also don't have a VirtualDub2 or the same Windows computer to test and see why you aren't getting any dropped frames when capturing with that.
The professional, external ones are more consistently able to do this because they are more robust and offer more control over the various levels and signals.
I fully agree! For this very reason, we use only these.
@JMY1000 I'm curious about the setup you have that's working. Are you saying that when you are using the Windows+VirtualDub2+Decklink Video Capture setup and are encoding the files as raw video/Lagarith/HuffYUV vs FFV1 that Decklink Video Capture isn't reporting any dropped frames (either during capture or in the logs) or that there aren't any at the same timecodes for the same tape (when using one of the other configurations) in the resulting files?
MIPoPS uses the Leitch DPS-575s (which are from the same line). I don't know all the engineering specifics either, but my basic understanding is that the external DPS is able to more consistently maintain a clean, stable signal from the deck to the computer, as well as identify/correct tape dropouts that are present on the tape (from breakdown/loss of magnetic coating).
Gotcha, thanks. Given that the dropouts aren't from breakdown/loss of coating and are instead an artifact of the unclean cuts present in the original recording, I'm still a little suspicious that a better DPS/TBC will be able to sufficiently clean-up the signal. I'll take a look at acquiring one though and giving it a shot.
I've noticed that some of the units seem to only have a few connectors on the back instead of the multitude I'd expect based on the manual; do you happen to know what's with that? Are these units just for managing the remote control for other units?
@privatezero / @libbyhopfauf: Regarding I/O with the Leitch DPS, which of these are you using?
@JMY1000 I'm curious about the setup you have that's working. Are you saying that when you are using the Windows+VirtualDub2+Decklink Video Capture setup and are encoding the files as raw video/Lagarith/HuffYUV vs FFV1 that Decklink Video Capture isn't reporting any dropped frames (either during capture or in the logs) or that there aren't any at the same timecodes for the same tape (when using one of the other configurations) in the resulting files?
When using Windows + VD2 + Decklink Video Capture, VD2 reports no dropped frames, only a few inserted frames. Unfortunately, VD2 doesn't generate a separate frame-by-frame capture log (only a general message log), but given that it isn't reporting any dropped frames, I don't think this is strictly necessary. I'm unfortunately unable to encode as FFV1, as this immediately crashes with VD2; raw video/Lagarith/HuffYUV all work fine.
@JMY1000 I'm not sure why any of those DPS units would come with only a few outputs (I would imagine it's a different model or version). The DPS-575s that we have at MIPoPS look exactly like the manual image.
@privatezero / @libbyhopfauf: Regarding I/O with the Leitch DPS, which of these are you using?
Analog video only into the DPS, analog video out from the DPS; analog video from DPS + analog audio from the deck into an Intensity Pro Analog video and analog audio into the DPS, SDI out from the DPS; SDI from the DPS into a Decklink If the former, have you needed to add an audio delay to compensate for any delay added by the DPS? If the latter, did your unit come with the audio module[s] preinstalled or did you have to install it aftermarket?
We use a combination of those two options. All of our analog decks are hooked up to the DPS. For video, the analog outputs are connected to the DPS; SDI from the DPS is routed to BlackMagic Express decklinks. The analog audio is connected from the decks to the DPS and then each DPS has an analog audio output routed to the BMEs.
Given that the dropouts aren't from breakdown/loss of coating and are instead an artifact of the unclean cuts present in the original recording, I'm still a little suspicious that a better DPS/TBC will be able to sufficiently clean-up the signal. .... When using Windows + VD2 + Decklink Video Capture, VD2 reports no dropped frames, only a few inserted frames.
Again, I am not familiar with the VD2, but if the artifacts that are showing up as dropped frames are present in the original recording, I would guess that instead of reading them as dropped frames, the VD2 is inserting those frames (whereas vrecord is dropping them without the assistance of an external DPS/TBC). Based on that screenshot though, it looks like it might still be resulting in a sync error as a result. But if there's no information on the tape at those parts, the DPS/TBC might still stabilize the connection. However, since like I said I haven't used the VD2.
We use a combination of those two options. All of our analog decks are hooked up to the DPS. For video, the analog outputs are connected to the DPS; SDI from the DPS is routed to BlackMagic Express decklinks. The analog audio is connected from the decks to the DPS and then each DPS has an analog audio output routed to the BMEs.
Gotcha, thanks. Have you found the need to set an audio delay or no?
Again, I am not familiar with the VD2, but if the artifacts that are showing up as dropped frames are present in the original recording, I would guess that instead of reading them as dropped frames, the VD2 is inserting those frames (whereas vrecord is dropping them without the assistance of an external DPS/TBC). Based on that screenshot though, it looks like it might still be resulting in a sync error as a result. But if there's no information on the tape at those parts, the DPS/TBC might still stabilize the connection. However, since like I said I haven't used the VD2.
Okay, thanks. That seems like it'd be a useful feature to have, no? I'm guessing that this would be a change that needs to happen upstream in ffmpeg? Is somewhere around here the correct place?
Gotcha, thanks. Have you found the need to set an audio delay or no?
No, not with our setup. But depending on other factors in the workflow, others might have found a need to.
Okay, thanks. That seems like it'd be a useful feature to have, no?
Unless it's part of the TBC/DPS processing or from digital-analog error concealment, I don't think that adding an artificial frame would technically qualify as an archival best practice. If that part is blank, it should stay blank and an external TBC/DPS would compensate accordingly.
Unless it's part of the TBC/DPS processing or from digital-analog error concealment, I don't think that adding an artificial frame would technically qualify as an archival best practice. If that part is blank, it should stay blank and an external TBC/DPS would compensate accordingly.
Fair enough. Ideally, it seems like the Blackmagic cards should be able to provide at least some of the signal processing functionality on their own so even partially-complete frames could be captured to the fullest extent possible, but it doesn't look like the Blackmagic API really is giving us a lot to work with. At least from what I see, the best you'd be able to do is hope that GetBytes()
returns something reasonable even if the bmdFrameHasNoInputSource
BMDFrameFlag
is set,
which without knowing much about the proprietary Blackmagic driver certainly isn't a guarantee.
class IDeckLinkVideoFrame : public IUnknown
{
public:
virtual long GetWidth (void) = 0;
virtual long GetHeight (void) = 0;
virtual long GetRowBytes (void) = 0;
virtual BMDPixelFormat GetPixelFormat (void) = 0;
virtual BMDFrameFlags GetFlags (void) = 0;
virtual HRESULT GetBytes (/* out */ void **buffer) = 0;
virtual HRESULT GetTimecode (/* in */ BMDTimecodeFormat format, /* out */ IDeckLinkTimecode **timecode) = 0;
virtual HRESULT GetAncillaryData (/* out */ IDeckLinkVideoFrameAncillary **ancillary) = 0;
protected:
virtual ~IDeckLinkVideoFrame () {}; // call Release method to drop reference count
};
That said, it's entirely possible that any other DPS doesn't do any better of a job—it's equally if not more of a black box—but you'd hope it does, given that's its primary purpose.
I'm a little tempted to just comment out the bit that actually skips frames without video and see what happens, but I'm a little scared of getting ffmpegdecklink
to build properly (even with the formula) 😅
@JMY1000 @retokromer @privatezero @libbyhopfauf @dericed it was very good discussions about dropped frames. I have exact same issue. when I captured over 1 hour. there is over 80 dropped frames and the sound at the end of video is out of sync. I use black magic design intensity 4 k . I route from VTR ( VHS )signal out composite to decklink as input . I captured by vrecord . it has many drop frames. is there any solutions for that ?? I tried to buy TBC but all of them is bad and there is no repair shop can fix mine . I have 2 DPS 575 but bad. is there any way to fill dropped frame by created frames to fill gabs and not effect by sync . or any have a solid solution for this I will appreciated. by the way, the funny of that, when I use a cheap usb capture ( easy cap ) I did not any issue of dropped framed .
@a7med12h I ultimately sprung for a DPS-470 which resolved my issues. I’d strongly recommend finding a repair shop for your DPS-575s if possible, it’s made my digitization experience much better.
While I wish vrecord/FFMPEG had a stopgap solution, AFAIK it still doesn’t. You could try to write your own (not that I had any success.) VD2 seemed to be more forgiving in terms of dropped frames, but I found its interface, options, and QC to be inferior to vrecord.
Best of luck!
amazing, but unfortunately I have been searching on a good repair shop can repair mine but I could not find. since I am not living in USA it is very difficult to find this. I want to ask you about DPS 475 , is it compatible with PAL and NTSC and SECAM ?
@JMY1000 about VD2 , I have tested it before . it is good . but I could not capture with MKV container. and also vrecord good to have monitor and logs which is very important and VD2 has not .
amazing, but unfortunately I have been searching on a good repair shop can repair mine but I could not find. since I am not living in USA it is very difficult to find this.
You may simply have to ship your unit to the US then. I know it won’t be cheap, but it’s probably cheaper than buying another.
I want to ask you about DPS 475 , is it compatible with PAL and NTSC and SECAM ?
The DPS-475 is NTSC only. Otherwise it’s identical to the DPS-575
@JMY1000 about VD2 , I have tested it before . it is good . but I could not capture with MKV container. and also vrecord good to have monitor and logs which is very important and VD2 has not .
Yup, that’s kinda unfortunately how it is. I couldn’t get it to work with FFV1/MKV either.
If you want, you could always look into the LD Decode project, but that’s frankly going to be far more difficult than just finding someone to repair your DPS-575s.
amazing, but unfortunately I have been searching on a good repair shop can repair mine but I could not find. since I am not living in USA it is very difficult to find this.
You may simply have to ship your unit to the US then. I know it won’t be cheap, but it’s probably cheaper than buying another.
I want to ask you about DPS 475 , is it compatible with PAL and NTSC and SECAM ?
The DPS-475 is NTSC only. Otherwise it’s identical to the DPS-575
@JMY1000 about VD2 , I have tested it before . it is good . but I could not capture with MKV container. and also vrecord good to have monitor and logs which is very important and VD2 has not .
Yup, that’s kinda unfortunately how it is. I couldn’t get it to work with FFV1/MKV either.
If you want, you could always look into the LD Decode project, but that’s frankly going to be far more difficult than just finding someone to repair your DPS-575s.
Then I have to find repair shop who can fix mine. Please please if you have any suggestions to send my DPS to shop can fix it . I will appreciate that.
@a7med12h I don’t know of any offhand, sorry.
I don't know why LD-decode was mentioned.
VHS-decode is what supports several tape formats now Sony 8mm included, and has become a defacto for VHS processing when it started beating up last generation time-base correctors based off of the adv chips even my FORA-310p is not worth maintaing and re-capping.
RF Capture directly is less painful than recapping any time base corrector these days honestly.
And over the last week did Sony 8 mm format docs have been updated by me rapidly, Umatic, Umatic SP, VHS, SMPTE-C are fully covered and production ready for decoding use for 99% of tapes.
For 8mm it's 1 signal 3 carriers (Audio/Video/RCTC) a very one run and done format with DV transfer for reference capture and sync, S-video for quality reference and FM RF for archival file.
I don't know why LD-decode was mentioned.
VHS-decode is what supports several tape formats now Sony 8mm included, and has become a defacto for VHS processing when it started beating up last generation time-base correctors based off of the adv chips.
I mentioned it since it’s the sorta-parent/upstream project; I’m aware that VHS-decode owns many other analog formats, but figured I’d start with the upstream side. But yes, for clarity @a7med12h, odds are that VHS-decode will have better support for your format.
Far less painful than recapping a time-based corrector honestly.
Depends on what you’re good at and how much money you have 😛. Honestly, I think for a beginner, “traditional” digitization is still a good route to go. Buying the Domsday hardware (or alternative), finding a compatible tape deck for every tape, installing a tap, and configuring the software still requires either a good bit of know-how or money. In @a7med12h’s case, it’ll probably be cheaper to get their units repaired if they can find someone. I’ll also point out that if it’s the capacitors, recapping them is pretty easy for any run-of-the-mill repair shop; they’re all pretty large, and the schematics included in the manual list specs for every component.
@JMY1000 Link the info not the rabbithole!
The tape/cvbs side is the core of current docs and clear info thats been entirely changed since early 2022, and vhs-decode has more scope then the ld-decode project and over 80 pages of docs dozens of tapped VCRs photo'ed and clear info on what to look for and how to tap its really not a obscure thing or concept info wise at this point now, alongside TBs of testing data for trial runs.
4 years ago it was a very arcane thing and that was due to the LD-Decode wiki being a dead end almost reserved for the DomesDay86 Project in technicality docs wise was missleading is all, the DdD is also very easy to have fabricated on demand with actual BOM formated to standards which is what held it back for years honestly.
But the reality is 30-50USD total setup 100USD at most the DdD is a nice to have but for tapes not really an absolute, the docs have been copy paste for the CX Cards for over 2 years now, and all decks are compatiable for video FM RF its standard test points for calibration if you can install a decklink/vrecord you can deploy decode in 2 hours or less on anything Umatic/VHS, Sony 8mm formats are a little limited but most of the backwards support Digital8 decks/camcorders have a jig point.
(The wiki also promotes self repair, i.e get the cheep and good tools today and learn slowly to save tons later, but never hurts to push the ENVE-lope in the world of archives😂)
@harrypm Great info, thanks. It’s be great watching VHS-decode progress, and it’s nice to see that there’s new stuff (even if I do need to catch up a bit it seems like.) Hadn’t heard about CX cards, those seem great, I’ll have to look into those myself.
That said, I think we may be a little off topic from my original issue; it might be worth redirecting this to another thread in the VHS-decode repo if you have more to add?
@JMY1000 CX Cards were hacked for raw since 2005, LaserDisc tinkering started in 2013, it was all lack of depth of docs holding it back last 2 years format support ramped up with multi CX Card capture in the same box etc its very real world scalable now. (even got the cheep RTLSDRs to do hifi-decode in real time)
Just wanted to correct the info here is all, the real world use has been published and well shown off but never hurts to spread public/enthusiast awareness in the context of archival from the more stable mediums, FM RF beats everything else out, even now has windows binarys ready for delivery with archives that was the turning point for saying its for anyone to play with.
There is a whole FAQ for the projects so I wont ramble on anymore haha.
@harrypm could u please tell me what is RF CAPTURE ? Do u mean the output one in VTR ? Is that for video+ audio🤕
@a7med12h Its capture and preservation of the RAW FM RF video signal as the heads read the tape, i.e directly after tracking on we capture the signal off test calibraton points, this applys to HiFi FM too, its format universal, after capture all the processing the VTR/VCR etc would do.
Time base correction, droput correction is all done software side, only one digtisation stage to file that can be processed entirely software defined 100% post flexiblity for stable media and a last chance to get everything possible off unstable media, no more external TBCs, no more messing around just raw data then FLAC compress down smaller then FFV1!
You normally also have a basic standard workflow running alongside for audio reference sync and hardware reference.
If you want to learn more look at the the links in above messages or the DD86 Discord is a better place to move this conversation really.
@harrypm thank u so much. I got clear idea. It is unbelievable
@harrypm I really like group of vhs decode , they are fantastic. Do you also group of archive digitization or AMIA community there
I’m currently attempting to digitize a number of Hi8 and Video8 tapes. These tapes are as-is from the camera; as such, there are imperfections in the originals recordings, including blank sections between clips and occasional damaged frames (particularly at the start of tapes and between clips). These discontinuities appear to be causing dropped frames—not ideal, but given that there’s not really much in there anyways to begin with, it’s something I’m okay with. Unfortunately, because of the high number of dropped frames—over 300 for certain tapes—the audio gets thrown out sync quite significantly. This makes the portions that are captured correctly borderline unusable.
I believe I’ve confirmed that the source of these dropped frames is the original media, as the frame drops happen at consistent points in the recording[s], and dropped frames aren’t exhibited when a different input is provided. Thus, my concern is primarily with fixing the audio sync. Right now, I see three possible options:
ffmpeg
, andbmdcapture
didn’t seem to be any better.I’d obviously like to stick with vrecord; is either the first or second option viable? If so, is this something that would happen entirely upstream (either in
ffmpeg
or inbmdcapture
), or is there something that should happen in vrecord? Is there another option I’m not seeing?Cases
Hardware
I/O constraints don’t appear to be an issue; the frame drops occur regardless what media I write to.
I saw #379; however, given I’m already using the unfiltered view and my deck has a TBC, I’m not sure it’s applicable.
Software
ffmpeg
Blackmagic Desktop Video
12.2.2, but 10.11.2 exhibited the same issues.
Capture settings
I’ve also tried using
bmdcapture
instead offfmpeg
; however, the issue persisted.Reference log
tape_ffv1_vrecord_input.log