search

JuliaIO / VideoIO.jl

Reading and writing of video files in Julia via ffmpeg
https://juliaio.github.io/VideoIO.jl/stable
Other
127 stars 53 forks source link

WIP: Move to BinaryProvider #141

Closed IanButterworth closed 5 years ago

IanButterworth commented 5 years ago

Attempting to make the build process more robust by moving to BinaryProvider.jl

staticfloat commented 5 years ago

Switching to BinaryProvider like this is not going to allow you to use preinstalled binaries. With BP, (unlike BinDeps) by default you can only use the binaries that are pointed to by the URLs in the build script. You would have to jump through a lot of hoops to get it to pick up the binaries from the environment, and that is by design.

On Fri, Feb 15, 2019 at 07:23 Ian Butterworth notifications@github.com wrote:

@ianshmean commented on this pull request.

In deps/build.jl https://github.com/JuliaIO/VideoIO.jl/pull/141#discussion_r257275778:

+const prefix = Prefix(get([a for a in ARGS if a != "--verbose"], 1, joinpath(@DIR, "usr"))) + +libpath = joinpath(@DIR, "usr/bin") #for forcing LibraryProduct to find libs in bin dir (as ffmpeg likes to do) +# These are the two binary objects we care about +products = Product[

  • ExecutableProduct(prefix, "ffmpeg", :ffmpeg),
  • ExecutableProduct(prefix, "ffprobe", :ffprobe),
  • ExecutableProduct(prefix, "ffplay", :ffplay),
  • LibraryProduct(libpath, [["libavcodec-ffmpeg.so.","libavcodec.","libavcodec.so.","libavcodec.ffmpeg.so.","avcodec-"].["53" "54" "55" "56" "57" "58"].[".dylib"]...], :libavcodec),
  • LibraryProduct(libpath, [["libavformat-ffmpeg.so.","libavformat.","libavformat.so.","libavformat.ffmpeg.so.","avformat-"].*["53" "54" "55" "56" "57" "58"]...], :libavformat),
  • LibraryProduct(libpath, [["libavutil-ffmpeg.so.", "libavutil.","libavutil.so.", "libavutil.ffmpeg.so.", "avutil-"].*["51" "52" "54" "55" "56"]...], :libavutil),
  • LibraryProduct(libpath, [["libswscale-ffmpeg.so.","libswscale.","libswscale.so.","libswscale.ffmpeg.so.","swscale-"].*["2" "3" "4" "5"]...], :libswscale),
  • LibraryProduct(libpath, [["libavfilter-ffmpeg.so.","libavfilter.","libavfilter.so.","libavfilter.ffmpeg.so.","avfilter-"].*["2" "3" "4" "5" "6" "7"]...], :libavfilter),
  • LibraryProduct(libpath, [["libavdevice-ffmpeg.so.","libavdevice.","libavdevice.so.","libavdevice.ffmpeg.so.","avdevice-"].*["53" "54" "55" "56" "57" "58"]...], :libavdevice), +]

The problem is that I haven't yet found linux platform-specific shared binaries, which would be ideal to control version like we're trying in windows and MacOS. Given linux typically(?) comes preinstalled with ffmpeg, the coverage of all the lib versions is needed for that. Perhaps there's an alternative that I'm missing..

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/JuliaIO/VideoIO.jl/pull/141#discussion_r257275778, or mute the thread https://github.com/notifications/unsubscribe-auth/AAH_aFvsQ8VoapTM9kDmFRlvyzG9aMEcks5vNtDTgaJpZM4a8_i2 .

IanButterworth commented 5 years ago

Ok, that makes sense. This branch is now working on Windows and MacOS, and should be ready for the linux binaries, so we can just leave it until the time is right.

IanButterworth commented 5 years ago

I just tried turning precompilation on for this and it shows the same error during testing as https://github.com/JuliaIO/VideoIO.jl/pull/114 where Testing IO reading... fails when attempting to use avformat_open_input

I've found some discussion on avformat_open_input crashing but perhaps someone more familiar with ffmpeg than me could advise? @kmsquire ? https://stackoverflow.com/questions/26552897/avformat-open-input-function-crashing

rdeits commented 5 years ago

Does precompilation work for this branch now that #143 has been merged?

Also, could we potentially get the linux FFMPEG binaries from https://github.com/jpsamaroo/FFMPEGBuilder (potentially with some additional updates to that repo)?

IanButterworth commented 5 years ago

@rdeits Indeed, precompillation is working in this branch now.

And I like the idea of grabbing the linux builds if they're within reach from FFMPEGBuilder.. I'm sure others are much more familiar than me with the issues around doing that than me

IanButterworth commented 5 years ago

Just to summarise. This PR is waiting for Linux ffmpeg binaries. I've not had time to create them yet, or explore getting them from https://github.com/jpsamaroo/FFMPEGBuilder

If anyone does, it would be great to merge this

jpsamaroo commented 5 years ago

I'm happy to transfer that repo wherever if people want to work on it; I don't have the bandwidth or desire to figure out the YASM build issues right now.

SimonDanisch commented 5 years ago

@jpsamaroo, can you move the builders to JuliaIO?

IanButterworth commented 5 years ago

How about this as an interim solution while the FFMPEG linux binaries aren't ready.. we build with Bindeps if Linux, and BinaryProvider if not linux?

IanButterworth commented 5 years ago

I've implemented an approach of BinDeps for Windows, BinaryProvider for everything else.

Side note.. Travis is passing but with a segfault during Cairo build on MacOS due to the ongoing Cairo/Homebrew issue https://github.com/JuliaPackaging/Homebrew.jl/issues/253 (which is in itself an example of why #60 is a good idea given Cairo is a dep through ImageView)

IanButterworth commented 5 years ago

Scratch that last comment, I finally got my head around FFMPEGbuilder via Travis (simpler than I expected) and got a fork releasing the linux binary (https://github.com/ianshmean/FFMPEGBuilder) so this PR should now support GNU linux, MacOS and Windows, if all tests pass.

jpsamaroo commented 5 years ago

In order for your build of FFMPEG to be decently capable and performant, the rest of the deps will also need to support all the same platforms, so there is probably still lots of work to do (unless you updated all of them without me noticing :) ).

IanButterworth commented 5 years ago

@jpsamaroo I just plucked out the GNU linux binary from FFMPEGbuilder, the windows and MacOS binaries came from: https://ffmpeg.zeranoe.com/builds/

Saying that, looking at Travis, the linux binary libs don't seem to satisfy BinaryProvider, which I guess means they're not dlopen-able, given they are present in the dir

SimonDanisch commented 5 years ago

Seems like libfdk-aac is missing, which leads to libavdevice to error, which makes it unsatisfied :(

SimonDanisch commented 5 years ago

Seems like it's already included (--enable-libfdk-aac --enable-nonfree):

  --enable-nonfree     \
  --disable-static     \
  --enable-shared      \
  --disable-debug      \
  --enable-avresample  \
  --enable-libass      \
  --enable-libfdk-aac  \

Do we actually need a library product for it, to not "loose it", or is there something else going wrong?

SimonDanisch commented 5 years ago

Ok, this should hopefully fix it :)

IanButterworth commented 5 years ago

Ah! @SimonDanisch I was rebasing too! 😂

SimonDanisch commented 5 years ago

Of course, I knew this would happen :D

IanButterworth commented 5 years ago

You do what's easiest. Especially if you're on a roll with getting this working!

SimonDanisch commented 5 years ago

continued in https://github.com/JuliaIO/VideoIO.jl/pull/159