search

JuliaIO / VideoIO.jl

Reading and writing of video files in Julia via ffmpeg
https://juliaio.github.io/VideoIO.jl/stable
Other
128 stars 53 forks source link

Major refactor of AV.jl #4

Closed kmsquire closed 10 years ago

kmsquire commented 10 years ago

I've done a major refactor of the AV.jl structure.

At this point, ffmpeg 2.2 and 2.3 and libav 0.8, 9, and 10 should be automatically detected and supported, although in this version, libav 0.8 and libav-10 have only had very minimal testing.

I've also added BinDeps support, but have only minimally tested on Linux.

If people could test this PR and post back with success, or open issues/PRs/commits to fix issues, that would be great. If there's nothing too major (or if there is no response), I'll commit and we can pick up the broken pieces later. ;-)

Other changes:

For the refactor, I've separated ffmpeg and libav libraries, and created one directory in each for each of the libraries (libavcodec -> AVCodecs, libavdevice -> AVDevice, etc.). In theory, these should be mostly compatible, but I was finding enough differences that I thought it best to start here.

Each of the library directories also functions as a submodule of AV, so that each module can be loaded separately.

The directory structure looks something like this:

src
src/ffmpeg
src/ffmpeg/AVCodecs
src/ffmpeg/AVCodecs/src
src/ffmpeg/AVCodecs/v53
src/ffmpeg/AVCodecs/v54
src/ffmpeg/AVCodecs/v55
src/ffmpeg/AVDevice
src/ffmpeg/AVDevice/src
src/ffmpeg/AVDevice/v53
...
src/libav
src/libav/AVCodecs
src/libav/AVCodecs/src
src/libav/AVCodecs/v53
src/libav/AVDevice
src/libav/AVDevice/src
src/libav/AVDevice/v53
src/libav/AVDevice/v54
...

At the top level, AV.jl is loaded. It detects the library versions for each library, which are made available to the submodules. Each submodule can then load the appropriate set of wrappers.

Right now, AVUtil, AVFormat, AVCodecs, and SWScale are loaded by AV.jl, which enables basic video functionality.

Additionally av_capture.jl provides a minimal wrapper, which is used by examples/playvid.jl.

Since the information in this comment isn't available elsewhere, I'm going to add it to the README.

Questions/comments/suggestions for changes, and most importantly, pull requests (or commits) with fixes and more functionality, are most welcome.

I'm planning to release an initial version after merging this.

timholy commented 10 years ago

If I run from the Unix command line, it works! This is truly awesome!

If I try to run interactively, I sometimes get a segfault; other times it works. This is with either the master or gtk branches of ImageView. Do you get the same thing? It's been working the last several times I tried it, so maybe there was something I inadvertently fixed, and at any rate that makes it hard to debug.

One other minor issue is that for me (just v0.3.0-rc1), somehow after loading all this code, sleep becomes wildly inaccurate:

julia> @time sleep(0.1)
elapsed time: 0.321868886 seconds (472 bytes allocated)

julia> @time sleep(0.1)
elapsed time: 0.987328383 seconds (472 bytes allocated)

julia> @time sleep(0.1)
elapsed time: 0.191481258 seconds (472 bytes allocated)

julia> @time sleep(0.1)
elapsed time: 0.835232123 seconds (472 bytes allocated)

In contrast, a fresh Julia session:

julia> @time sleep(0.1)
elapsed time: 0.112293411 seconds (63004 bytes allocated)

julia> @time sleep(0.1)
elapsed time: 0.10120421 seconds (472 bytes allocated)

julia> @time sleep(0.1)
elapsed time: 0.101212995 seconds (472 bytes allocated)

julia> @time sleep(0.1)
elapsed time: 0.101203731 seconds (472 bytes allocated)

This isn't your problem, as simply saying using ImageView is enough to trigger it.

kmsquire commented 10 years ago

Great, Tim! I'm glad to hear that we're heading in the right direction.

I do see segfaults sometimes, worse with some library versions. With the latest ffmpeg, the only time I see crashes is if I press ctrl-c while playing a video.

I tried debugging with debug builds of both Julia and libav, but the crashes went away.

I haven't kept track recently--do you know if interrupts are disabled for all ccalls? I think probably not, so I should try that, at least for key functions.

Anyway, I'll merge with master a little later, and see if I can't fix some of these crashes. Cheers!

kmsquire commented 10 years ago

Okay, I've added sigatomic around sws_scale, which is the function that does the conversion from yuv to rgb, and I haven't had any crashes in minor testing with libav 0.8, 9, and ffmpeg 2.3.

Also fixed a few other bugs and added minor functionality along the way.

@timholy, when you get the chance, can you pull and test again? If all looks good, I'll merge to mainline.

timholy commented 10 years ago

In a couple of tries, I could not make it segfault. Yay!

kmsquire commented 10 years ago

Perfect! I'll merge with master.