Open emk opened 8 years ago
Yeah, right now all I do is support the latest version, which is currently 2.8.2, it's really hard to support every version available, especially when all I'm doing is linking dynamically without touching the headers.
I did provide some features that build the latest ffmpeg and link it in statically, but there are some standing Cargo bugs that make it not work.
Is there a recommended way to build/supply 2.8.2 on systems with an older version of ffmpeg or libav?
(Also, this was my first debugging session with rr
, and I'm suitably impressed.)
If you're on Ubuntu, you could look for some ppa that provides the latest, otherwise you can always build it from source and install it in /usr/local
.
The final ideal target would be to just pass as features static
and build
, and everything works magically, but I've had problems making that work, at least for running tests.
It does build it but Cargo seems to treat it as an .rlib
instead of .a
, so it breaks on linking.
OK, more experimentation reveals:
error: could not find native static library
avutil, perhaps an -L flag is missing?
.I used the following dependencies, both with and without static
:
[dependencies.ffmpeg-sys]
version = "2.8.0"
default-features = false
features = ["static", "avcodec", "avdevice", "avfilter", "avformat",
"swresample", "swscale"]
optional = true
[dependencies.ffmpeg]
#version = "~0.2.0"
path = "rust-ffmpeg"
default-features = false
features = ["static", "codec", "device", "filter", "format",
"software-resampling", "software-scaling"]
optional = true
At this point, I'm not sure that there's any way to get this working on Ubuntu 14.04 LTS without asking my users to build ffmpeg from source, which I'd just as soon avoid (especially since the version number needs to be a precise match, and I don't want to build it anywhere other software might find it).
One possible solution would be for ffmpeg-sys
to build its own copy of ffmpeg. Another possible solution would be to include a supplemental wrappers.c
file that added tiny C wrappers for struct fields, so instead of writing stream.metadata
, the code could write ffmpeg_wrapper_stream_get_metadata(stream)
. This might make it possible to support a wider range of ffmpeg versions, at least if you never needed to allocate any of the structs on the stack.
For my needs, the best solution may be to write a thin wrapper around the CLI tools. :-( Many of them can now output JSON, at least, and it would be easier for my users to get building.
You don't need to define features for both ffmpeg-sys
and ffmpeg
, ffmpeg
features define the required ones automatically, basically this.
[dependencies.ffmpeg]
optional = true
features = ["static", "build"]
That's all you should need, that will make ffmpeg-sys
build ffmpeg from sources.
Can you try it out with your binary and see what happens?
@emk Welcome to the wonderful world that is the ffmpeg API. I had many wonderful hours wtih it :D
I actually spend my weekend getting ffmpeg to build statically and get a *.so out of my library that uses rust-ffmpeg
but with static ffmpeg dependencies. In the end I triumphed. I can reduce that to a minimal project to showcase what I did (as soon as I have some time to spare the next few days).
But in the end it boils down to building ffmpeg with https://github.com/zimbatm/ffmpeg-static (with modifications) and overriding the rust-ffmpeg-sys
search path. See https://github.com/meh/rust-ffmpeg-sys/pull/11.
@lummax ideally I'd like to have ffmpeg-sys
doing all that crap to build statically.
I also need to include ffmpeg-sys
directly so that I can access the loglevel options, unfortunately.
Thank you all for your help trying to get ffmpeg
built! I'm probably going to go in another direction for now, but I'll probably try again later at some point.
@meh Me too. But there is a lot of complexity.
You want 'H264'? Get a static version of libx264!
And rust-ffmpeg-sys
can not deal with every dependency. In the end the library gets too big. I think it's sensible to let the user deal with that (Especially with regards to linux distros that maybe distribute static versions of the required dependencies).
@emk Just implement the logging stuff and open a PR :D
@emk I can work on the logging stuff if you need it, just tell me what you need safely wrapped that isn't present yet and I'll bump it on the backlog.
Me and @lummax have just been working on what we personally needed so far.
@lummax it shouldn't make it any more complex, it will just make the build.rs
a little more complex, and I'm fine with it since it's all it has to do.
@lummax to expand on that, there are already Cargo features that check what libraries you want to compile in, it's just a matter of checking for them and downloading/building them statically.
As promised, here you have it: https://github.com/lummax/rust-ffmpeg-static
Although I'd also like this complexity to be included in rust-ffmpeg-sys
, I don't think it is the right place. The complexity (and rather long build times) are hidden, aswell as the possible tradeoffs (this linked repository disables the hardware acceleration).
I think it would be best to include an example or a showcase similar to mine in the README and let people figure this out on their own.
@saeschdivara This might also concert you as https://github.com/meh/rust-ffmpeg-sys/issues/14 touches this issue.
They wouldn't be hidden, you'd have to opt-in with the build
feature like it is already.
I'll give it a shot later.
Heads up on this, now the build script checks headers for ifdefs, so it should work.
As in, when it's not building ffmpeg itself it will check the headers on the system to figure out the configuration.
Nice! I ultimately ended up talking to the ffmpeg command-line tool via a batch job wrapper, which works well enough for my use cases (probing, output in various formats). But thank you for working on this! If I ever outgrow the CLI interface, I may be back. :-)
I'm not sure if there's anything you can do about this one, but I thought I would mention it. If you really want to reproduce this locally, I'm happy to provide source code and step-by-step repro instructions. But mostly I'm just sharing this because:
#[repr(C)] struct
definitions in the FFI.I began by adding a
metadata
method toStream
:But when I tried to iterate over stream metadata, my program crashed. So I decided it was time to try out the
rr
debugger, which records execution traces, and which provides support for stepping backwards in GDB. So I downloaded it, and captured a recording of the crash:Executing up to the crash and then rewinding back out of
av_dict_get
revealed some funny business:Wait, what?
self.ptr
is 0xfb000000df? How did I get a metadata dictionary pointer that looked like 0xfb000000df? That can't be right.So I set a hardware watchpoint on the bogus pointer, and run backwards until I see that pointer being set:
Repeating this process several times eventually brings me to:
So the bogus pointer in
self.context.ptr.streams[0].metadata
is being created byav_reduce
inside ofavformat_open_input
. But wait,av_reduce
performs calculations with rational numbers. That shouldn't have anything to do withmetadata
. But let's take a look atAVStream
again:Yup.
metadata
is right between two rational fields. Time to compare the Rust structure:To the C version on my local system:
Oooh, what's this
FF_API_R_FRAME_RATE
thing? Let's grep a bit:Bingo. So now we know that
ffmpeg-sys
requires at leastLIBAVFORMAT_VERSION_MAJOR
55 to work, but Ubuntu 14.04 only ships 54. And with 54,ffmpeg-sys
will look for themetadata
field wherelibavformat
has actually storedavg_frame_rate
. Which explains why backwards tracing with hardware breakpoints thinks that ourmetadata
pointer was created byav_reduce
.pounds head on desk
I don't have any good answers here.