Backend discussion

[kmsquire]

• [x] Explore avbin
• [x] BinDeps integration
• [x] Support more recent versions of libav/ffmpeg
• [x] At load time, determine which wrapper version to load.
• [x] Probably switch to using StrPack for everything but reading/decoding frames. Dealing with struct pointers directly through Julia is rather painful right now, and some of the larger structs created by Clang ( e.g., immutable Array_Uint8_1024) take way too long to compile and instantiate. (I can provide more details if needed).
• [ ] Clean up and remove unnecessary functions and types. Right now, almost everything is included, because that was easiest to get started, but large chunks are probably not needed.

cc: @ihnorton @lucasb-eyer @timholy

[lucasb-eyer]

I think the gist of avbin is well stated at the end of their readme:

The AVbin dynamic library exports all of Libav's functions from libavcodec, libavutil, libavformat, and libswscale. It also exports some higher-level functions which have a fixed ABI (they will not change in incompatible ways in future releases), documented in include/avbin.h.

From what I've seen, they have the right approach of tackling the "problem" of integrating libav*. Possible disadvantages:

• AVbin/AVbin#54 seems important but has gotten no reaction in almost a year.
• It looks like a one-person effort, it seems to have been dead for 3 years already once in the past. (2008-2011)
• Binaries available "only" for Linux, Mac OS X and Windows. Needs to be compiled for other systems, but how portable is it? Actually, does Julia run on any other systems? An option would be to fall-back to the system's libav* and pray for a compatible version.

That's all I could see, but note that I have zero experience with distributing Julia packages. Unfortunately, I won't have the time to give it a try myself for at least a month.

[timholy]

The code in AVBin's src is not huge, and my experience is that binary dependencies are a pain that is compensated only if they provide considerable add-on value.

I would personally recommend targeting libav directly. Then it becomes Julia's job to provide a nice, stable interface, perhaps inspired by how AVBin does it. But I can easily be persuaded otherwise.

[timholy]

With regards to BinDeps integration, it's nice that the binaries for Windows are in 7z format. That should make it pretty easy. I sure wish that pages like this provided a link titled latest or stable. If we want to keep up-to-date, I fear we'll have to parse the HTTP on that page. OTOH, they clearly keep old versions around for a while, and there's some merit in choosing a consistent version to install.

On OSX, I don't know whether we'd need to engage HomeBrew or not.

[lucasb-eyer]

Now I'm confused. (I don't really know BinDeps yet.) Your link for libav is a binary dependency, too? AVBin is a single libavbin.so/dll which includes libav*, so all symbols you get from libav* are in there, in addition to the stable API, so there's nothing stopping us from providing a different, Julian API. Not that I'm fixed on AVBin (the issue I linked to might be a showstopper!) but I don't understand the difference here?

[timholy]

I only found AVBin's GitHub page, didn't realize they provided binaries too. I thought we would have to compile & host it, which would be annoying.

[timholy]

Although those binaries are not exactly fresh, and for video formats I wonder if we need something that stays up-to-date?

[lucasb-eyer]

I'm really not sure either.

Why I'm not convinced of going straight for libav is that either a) we'll have to provide binaries known to work, hence do exactly what AVBin does, or b) hope that the user's system-provided libav is the version we expect. It might be that in practice b) is not as bad as I imagine. (We're stuck with Ubuntu 12.04 at work and most scientists I know aren't the "I'll take the time to compile the correct version and all its dependencies locally and adapt LD_LIBRARY_PATH" kind.)

From what I see in core Julia, it looks like the philosophy is to bundle dependencies.

[kmsquire]

Okay, catching up here.. I'm going to answer individual parts in separate posts.

I agree that AVBin seems like a good idea, but I am also concerned about the lack of development there, and the lack of progress on the issue that Lucas pointed out.

On the other hand, it is nice that it provides a uniform interface.

Before I knew about AVBin, my plan was to do something similar: provide a stable interface in each subdirectory corresponding to the different versions of libav/ffmpeg (the gratuitous interface changes from version to version are a bit annoying), and include the version corresponding to the version installed on the user's system. I don't know exactly how much work this would entail, but a canned version of this really starts to be more appealing.

[timholy]

I certainly have no objections if folks want to go this way---just wanted to make sure that my frustrations with ImageMagick didn't go in vain. OTOH, libhdf has been pretty seamless, so perhaps I shouldn't take that too seriously myself.

[kmsquire]

I'm actually not convinced about AVBin yet either. For example, if we want to support as many image codecs as possible, libav/ffmpeg (and therefore AVBin) would have to be compiled on the user's system, because the combination of licenses for the various codecs makes the resulting library undistributable.

[kmsquire]

Anyway, I'll try to take a closer look at AVBin today, and see if it is worth using, or if it could be used for inspiration, as Tim suggested.

[kmsquire]

Just a quick update here...

For a project I'm working on, I needed to wrap the latest ffmpeg. Surprisingly, I found that I needed very few changes to get the tutorial to work across libav 0.8 and ffmpeg 2.2.

I also wrapped avbin. This was doable, and about as painful as wrapping libav itself. Because the interface to avbin is somewhat limited (for my uses), I would need to wrap (parts of) libav directly anyway, so my inclination is to just wrap the original library. My experience above suggests this actually won't be too bad.

To do (not necessarily in this order) (copied above):

1. Provide wrappers for different versions of ffmpeg/libav (version 0.8, 0.9, and 0.10 for libav). I believe it possible to only provide three wrappers. Each set of wrappers will go in its own subdirectory.
2. At load time, determine which wrapper version to load.
3. Probably switch to using StrPack for everything but reading/decoding frames. Dealing with struct pointers directly through Julia is rather painful right now, and some of the larger structs created by Clang ( e.g., immutable Array_Uint8_1024) take way too long to compile and instantiate. (I can provide more details if needed).
4. Clean up and remove unnecessary functions and types. Right now, almost everything is included, because that was easiest to get started, but large chunks are probably not needed.
5. BinDeps support

[timholy]

Cool stuff! Really amazing.

[kmsquire]

The ffmpeg-2.2 wrapping is now available in the ffmpeg branch. I'm going to try to work on integrating things so others can test this easier.

[kmsquire]

Sorry, ffmpeg-2.2 branch.

[kmsquire]

Most of this work was done in #4 and merged. The load time is still significant, so I want to work further on cutting out unneeded functions (of which there are likely many).

[IanButterworth]

Doing a bit of a tidy up, so I will close this. Fun to read back the early discussions :)