search

georgmartius / vid.stab

Video stabilization library
http://public.hronopik.de/vid.stab/
Other
859 stars 109 forks source link

vidstabdetect appears to be nondeterministic? #111

Open egnor opened 3 years ago

egnor commented 3 years ago

Hi, if I run vidstabdetect repeatedly this video clip (which is not special as far as I know)...

https://user-images.githubusercontent.com/279819/122711928-e7cd3c00-d217-11eb-8bfe-aae4530a0636.mp4

ffmpeg -i clip.mp4 -filter:v vidstabdetect -f null -

...the output transform.trf differs from run to run, as compared with "diff" (though the "show=1" output is identical). Is this expected?

It's possible the nondeterminism is in video decoding but that seems unlikely -- asking ffmpeg to transcode the same clip (with no vidstabdetect) produces identical output. The transforms seem to be effectively very similar, but vidstabtransform result also produces slightly different results, which is inconvenient in a variety of ways for managing a pipeline.

georgmartius commented 3 years ago

Mh, I cannot remember putting any source of non-determinism in it. How is the trf file different? (Are you still using the ascii format?)

egnor commented 3 years ago

transforms.1.trf.txt transforms.2.trf.txt

(Didn't even realize there was a non-ASCII format!)

egnor commented 3 years ago

Aha, if I set OMP_NUM_THREADS=1 then output becomes deterministic!

Probably it's the thread-parallelized loop in calcTransFields, causing the localmotions to be output in nondeterministic order. Might be worth sorting those on output? (And similarly elsewhere?)

georgmartius commented 3 years ago

Okay, makes sense. Why is it important for you that the outputs are the same. In the next step, the order of the fields in the transform file does not matter.

egnor commented 3 years ago

Sadly, the final video I get is not the same. But it seems that the transforms.trf difference is a red herring! Even with transforms.trf identical, two different runs of vidstabtransform on the same source give different results. :disappointed:

% ffmpeg -i clip.mp4 -vf vidstabtransform clip_stab.1.mp4
...
% ffmpeg -i clip.mp4 -vf vidstabtransform clip_stab.2.mp4
...
% diff clip_stab.1.mp4 clip_stab.2.mp4
Binary files clip_stab.1.mp4 and clip_stab.2.mp4 differ

Unfortunately, setting OMP_NUM_THREADS=1 doesn't fix this one.

As for "why do I care": Broadly speaking, it's valuable to have reproducible build steps at each point of a pipeline. Build pipeline systems often work better and more efficiently when equivalent results are bitwise identical, so they can skip subsequent steps. It also allows other people to verify or reproduce builds, which is important for security (ensure nothing was tampered with), debugging (reproduce build results to understand them), etc.. See reproducible-builds.org, martinfowler.com/bliki/ReproducibleBuild.html, en.wikipedia.org/wiki/Reproducible_builds and the like -- most of which are talking about software builds, but the same concepts apply to other artifacts.

In my very specific case, I have scripts that turn "art sources" into production videos, which then get checked in if they've changed. If the production video changes every time the pipeline is run, then there's a lot of churn and unnecessary pushing.