JuliaIO / VideoIO.jl

Reading and writing of video files in Julia via ffmpeg
https://juliaio.github.io/VideoIO.jl/stable
Other
125 stars 53 forks source link

Enable adjustable codec multithreading #332

Closed IanButterworth closed 2 years ago

IanButterworth commented 2 years ago

I had assumed that ffmpeg/h.264 automatically handled multithreading, but it seems not to be the case. The small change in this PR is based on https://stackoverflow.com/questions/43251612/ffmpeg-how-to-use-multithreading

I propose that the default is set to Threads.nthreads() but that users can tune it as they wish.

Decoding

Master

julia> function foo()
    f = openvideo("vid.mp4")
    i = 1
    img = read(f)
    i += 1 
    while !eof(f)
        read!(f, img)
        i += 1
    end
    close(f)
    @show i
end

julia> @time foo();
i = 1789
 27.231079 seconds (1.91 k allocations: 9.032 MiB, 0.03% gc time)

This PR

Where Threads.nthreads() == 16

julia> @time foo();
i = 1789
  3.339067 seconds (1.91 k allocations: 9.032 MiB)

Encoding

Not much of an improvement in my case, but might be tunable for a given system

Master

julia> imgstack = map(_->rand(UInt8, 100,100), 1:1000);

julia> @time VideoIO.save("vid.mp4", imgstack); # what master has. Assumed to do something auto
  2.168394 seconds (30.54 M allocations: 638.736 MiB, 3.87% gc time, 6.37% compilation time)

This PR

julia> @time VideoIO.save("vid.mp4", imgstack); # defaults to thread_count = Threads.nthreads() == 6
  2.025728 seconds (30.01 M allocations: 610.917 MiB, 4.35% gc time)

Saves about 1 minute (6 -> 5 mins on ubuntu) in CI with only 2 threads

codecov[bot] commented 2 years ago

Codecov Report

Merging #332 (cfba676) into master (bbc0d6e) will increase coverage by 0.03%. The diff coverage is 100.00%.

:exclamation: Current head cfba676 differs from pull request most recent head 40d022e. Consider uploading reports for the commit 40d022e to get more accurate results Impacted file tree graph

@@            Coverage Diff             @@
##           master     #332      +/-   ##
==========================================
+ Coverage   81.57%   81.60%   +0.03%     
==========================================
  Files          17       17              
  Lines        1183     1185       +2     
==========================================
+ Hits          965      967       +2     
  Misses        218      218              
Impacted Files Coverage Δ
src/avio.jl 81.72% <100.00%> (+0.04%) :arrow_up:
src/encoding.jl 93.23% <100.00%> (+0.03%) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update bbc0d6e...40d022e. Read the comment docs.

IanButterworth commented 2 years ago

thread_count is now adjustable on both encode and decode, but on encode defaults to the codec default given I didn't see any improvement with increasing that. However on decode it's a 10x improvement for my system (nothing sets it to the codec default i.e. previous behavior)

julia> @time VideoIO.load("vid.mp4");
  0.218441 seconds (904 allocations: 286.130 MiB, 34.51% gc time)

julia> @time VideoIO.load("vid.mp4", thread_count = nothing);
  2.203609 seconds (904 allocations: 286.130 MiB, 0.17% gc time)