Getting MP4 files to work on windows/linux

[MaximilianWang]

Hello,

So after some elbow grease I was able to get this library incorporated into my project and is happy to see that I can play OGV files on windows and MP4 on android. I am currently trying to make an app that will need to be able to play MP4 files on android, windows, and linux.

I have tried and failed to get MP4 files to play on Windows so either I am doing something wrong or MP4 is currently not supported. If that is the case how do I go about getting this library to play MP4 files on windows/linux?

Regards,

N

[RBogie]

The Windows and Linux versions are using ffmpeg for playing video files. The prebuild versions of ffmpeg currently only support ogg/vorbis and webm's based on VP8 or VP9. The reason for this is licensing. MP4 files mostly use the H264 codec, which is bound by licensing. Since we are no lawyers, (And don't have to money to pay lawyers), we decided to not include those codecs in the ffmpeg build.

Now, if you want to use these formats, you'll have to do the following: Firstly, you'll need to check the licenses, and make sure you fullfill all requirements. Then, you have to compile a new gdx-video-desktop-natives.jar. You'll have to edit the configure options for ffmpeg which are in the build-ffmpeg-{platform}.xml files in the Desktop/jni folder. You can then compile by running ant -f build.xml in the same folder. Note that you need to run these on a linux computer, and for the windows build you'll need mingw-w64. FFMpeg compile will also ask for a recent assembler like yasm or nasm.

Good luck!

[MaximilianWang]

Hello,

Thank you for the quick response. Since I am not too familiar with these things I am afraid I'll have to beseech you to answer a few more questions. Please excuse me for my ignorance.

By the editing of the config files, are you talking about the flags like '--enable-decoder=vp9 --enable-decoder=theora' in the xml files and I just need to add something like '--enable-decoder=hevc' or '--enable-decoder=mp4' (referring to decoders section at http://ffmpeg.org/ffmpeg-codecs.html#Video-Decoders). Is there any other things I need to change in the xml files?

Also regarding your instructions for building, please confirm if I got the general gist of it:

1. Pull the project, edit all the xml files to add the correct flag (not sure what the exact flag is for mp4. Is it just mp4?).
2. Obtain Windows system, install java jre, jdk, ant, mingw=w64, and yasm. Run ant -f build-ffmpeg-[platform].xml for both windows32 and 64 xml file.
3. Obtain linux system, install java jre, jdk, and ant. Transfer the project from windows system to linux system. Run ant -f build-ffmpeg-[platform].xml for both linux32 and 64 xml files.
4. Run ant -f build.xml on same linux system. Take resulting .jar file and include it into the 'desktop' module of the LibGDX project.

Finally regarding the resulting 'gdx-video-desktop-natives.jar', I have already tried to use the compiled one you provided in one of the other issues and included it into the 'desktop' project however when I try to run the project I would get errors like "Class not found: com.badlogic.gdx.video.VideoPlayerDesktop" so instead I have just took the source of the desktop module and copied it into the src folder inside the 'desktop' libgdx module to get it working. This does not sound right so can you tell me what I am doing wrong?

Regards,

N

[RBogie]

Indeed, the flags you are mentioning are the ones that should be adapted. I think (Not sure) that you would need decoder h264 and demuxer matroska. Since matroska is already included, you would only have to add the --enable-decoder=h264 flag for each platform.

Now, the platform building is as follows: The Windows and linux builds both have to be performed on a linux computer. This computer must have yasm/nasm and mingw-w64. Obviously, it also needs the normal gcc compilers (multilib versions if you want to compile for both 32 and 64bit (Just make sure your gcc has the -m32 and -m64 flags available)). If you want to compile for mac, you'll need a mac. The scripts for mac are not yet tested though (This will be tested soon).

The gdx-video-desktop-natives.jar needs to be on the classpath. This means that when using maven or gradle, it is automatically added as a dependency for the desktop project. You can also just supply it as a classpath parameter. If you're trying to include it manually, you would indeed have to copy the contents from the jar into your project.

[MaximilianWang]

Hello,

So I deleted the java source code from the desktop module of this project that I have copied into my local project and I have added the gdx-video-desktop-natives.jar to the classpath in the gradle build file for desktop module (+compile files('libs/gdx-video-desktop-natives.jar' in dependancies section) and I still get the same Class not found errors.

I have successfully been able to ant build for the linux 32, linux 64, and windows 32 platform ffmpeg build files but am having troubles with the windows 64 one. I have installed the gcc multilib, mingw-w64, and the entire list of dependancies mentioned in https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu but It still wont build. The bottom of the build logs says something like:

 `Buildfile: /somewhere/gdx-video/gdx-video-desktop/jni/build-ffmpeg-windows64.xml

precompile:

compile: [exec] install prefix ../../libs/windows64 [exec] source path /somewhere/gdx-video/gdx-video-desktop/jni/ffmpeg/ffmpeg-2.2.2 [exec] C compiler x86_64-w64-mingw32-gcc [exec] C library mingw64 [exec] host C compiler gcc [exec] host C library glibc [exec] ARCH x86 (generic) [exec] big-endian no [exec] runtime cpu detection yes [exec] yasm yes [exec] MMX enabled yes [exec] MMXEXT enabled yes [exec] 3DNow! enabled yes [exec] 3DNow! extended enabled yes [exec] SSE enabled yes [exec] SSSE3 enabled yes [exec] AVX enabled yes [exec] XOP enabled yes [exec] FMA3 enabled yes [exec] FMA4 enabled yes [exec] i686 features enabled yes [exec] CMOV is fast yes [exec] EBX available yes [exec] EBP available yes [exec] debug symbols yes [exec] strip symbols yes [exec] optimize for size no [exec] optimizations yes [exec] static yes [exec] shared no [exec] postprocessing support no [exec] new filter support yes [exec] network support no [exec] threading support w32threads [exec] safe bitstream reader yes [exec] SDL support no [exec] opencl enabled no [exec] libzvbi enabled no [exec] texi2html enabled no [exec] perl enabled yes [exec] pod2man enabled yes [exec] makeinfo enabled yes [exec] [exec] External libraries: [exec] [exec] Enabled decoders: [exec] hevc vorbis vp8 [exec] theora vp3 vp9 [exec] [exec] Enabled encoders: [exec] [exec] Enabled hwaccels: [exec] [exec] Enabled parsers: [exec] [exec] Enabled demuxers: [exec] matroska ogg [exec] [exec] Enabled muxers: [exec] [exec] Enabled protocols: [exec] file [exec] [exec] Enabled filters: [exec] aformat atrim null [exec] anull deshake setpts [exec] aresample format trim [exec] [exec] Enabled bsfs: [exec] [exec] Enabled indevs: [exec] [exec] Enabled outdevs: [exec] [exec] License: LGPL version 2.1 or later [exec] Creating config.mak, config.h, and doc/config.texi... [exec] config.h is unchanged [exec] config.asm is unchanged [exec] libavutil/avconfig.h is unchanged [exec] CC ffprobe.o [exec] LD ffprobe_g.exe [exec] CP ffprobe.exe [exec] STRIP ffprobe.exe [exec] x86_64-w64-mingw32-strip:stuyc9ZW: Protocol error [exec] make: *\ [ffprobe.exe] Error 1

BUILD FAILED /somewhere/gdx-video/gdx-video-desktop/jni/build-ffmpeg-windows64.xml:33: exec returned: 2 `

Any ideas?

Regards,

N

[RBogie]

I haven't seen that error before. From what I've seen before, is that ffmpeg sometimes needs additional flags for encoders on different platforms. However, this is mostly trial and error. I've not seen any documentation on this unfortunately.

However, it does look a bit like it's reusing some files. Could it be that it's trying to build in the same folder as where the windows32 executables were build? Are you running through build.xml, or are you manually configuring ffmpeg?

[MaximilianWang]

Hello,

Unless I misinterpreted your initial instructions, I was running ant -f build-platform-ABC.xml for each of the xmls before finally doing ant -f build.xml but I keep getting stuck at windows64 build xml. Please don't tell me that I am only supposed to run build.xml and build.xml will run the relevent xml files itself.

Regards,

N

[RBogie]

doing an ant -f build.xml runs the linux and windows xmls in the correct order. However, if the ffmpeg-windows64 build fails during manually running it, without having ran the windows64.xml or the build.xml first, there's something else going wrong.

For now, try removing all the build directories, and just running ant -f build.xml. It might just solve everything ;)

[MaximilianWang]

Hello,

Thanks for the assistance so far. Sadly after cleaning the build destination, the build-ffmpeg-windows64.xml still refuses to build. Not sure if it could be a cause but I am building these xml files on a Ubuntu 14.10 system running on virtualbox.

Regards,

N

[MaximilianWang]

Hello,

So after installing a whole bunch of things through apt-get the ant build for build-ffmpeg-windows64.xml is finally finishing. Now I have having issues doing the ant build with build-linux64.xml with the following error:

Buildfile: /somewhere/gdx-video/gdx-video-desktop/jni/build-linux64.xml

precompile:

create-build-dir: [copy] Copying 6 files to /hsomewhere/gdx-video/gdx-video-desktop/jni/target/linux64

compile:

link: [exec] /usr/bin/ld: ../../ffmpeg/libs/linux64/lib/libavcodec.a(hevc_cabac.o): relocation R_X86_64_PC32 against symbol `ff_h264_cabac_tables' can not be used when making a shared object; recompile with -fPIC [exec] /usr/bin/ld: final link failed: Bad value [exec] collect2: error: ld returned 1 exit status

BUILD FAILED /somewhere/gdx-video/gdx-video-desktop/jni/build-linux64.xml:136: exec returned: 1

I have cleaned the build directory ffmpeg and it made no difference.

Regards,

N

[RBogie]

Good to hear that the windows build is now working. The ffmpeg build done in this library is not completely standard. Because we want to link it in another DLL, the library is compiled as position independent code. To do this, we have to patch ffmpeg before building. Your error is related to this process of building. Could it be that you accidentally removed -fpic from the arguments? If not, this means that the patch is not compatible with h264 for some reason. If this is the case, I'm afraid I won't be able to help you any further. Maybe you can find the issue on Google.

Good luck!

[MaximilianWang]

Hello,

So if I just want to get win32, win64, linux32 working do I just remove the lines related to linux64 out from the build.xml and then generate a Jar from that? I and still having issues including a Jar filled with .dll and .so files into gradle via the compilefiles (path) command.

Regards,

N

[RBogie]

By removing Linux 64 from the XML files, it would stop building this one.

The native jar has to be added to your maven/cradle local storage. Otherwise it won't be resolved. Normally the build server would do this and deploy to maven central. As soon as the build server does, gradle would resolve it just fine. On Feb 10, 2016 08:30, "NauE5" notifications@github.com wrote:

Hello,

So if I just want to get win32, win64, linux32 working do I just remove the lines related to linux64 out from the build.xml and then generate a Jar from that? I and still having issues including a Jar filled with .dll and .so files into gradle via the compilefiles (path) command.

Regards,

N

— Reply to this email directly or view it on GitHub https://github.com/libgdx/gdx-video/issues/22#issuecomment-182237220.

[MaximilianWang]

Hello,

I see, I guess I will have to have a look at other resources to see how I can get gradle to extract the right .so or .dll from the jar and put it in right position. Please excuse me but I am quite unfamiliar with working with native libraries in Java.

What would be a valid way to test if the project even properly included the native libs into its class path (based upon the jar file you provided in #2 and what I will generate using build.xml after i remove linux64 stuff) because so far all I did was take the com.badlogic.gdx.video package from your master branch and directly copy it into the desktop module of my project and I was able to play ogv videos.

Regards,

N

[RBogie]

If you're building your own runnable jar, you can try to open the jar with a zip archiver. If it has the native files in the root of the jar, it's working. If you're just adding the jar to your classpath in the classpath parameter, I think the only way to see if it works is by trying to run it.

2016-02-10 13:45 GMT+01:00 NauE5 notifications@github.com:

Hello,

I see, I guess I will have to have a look at other resources to see how I can get gradle to extract the right .so or .dll from the jar and put it in right position. Please excuse me but I am quite unfamiliar with working with native libraries in Java.

What would be a valid way to test if the project even properly included the native libs into its class path (based upon the jar file you provided in #2 https://github.com/libgdx/gdx-video/issues/2 and what I will generate using build.xml after i remove linux64 stuff) because so far all I did was take the com.badlogic.gdx.video package from your master branch and directly copy it into the desktop module of my project and I was able to play ogv videos.

Regards,

N

— Reply to this email directly or view it on GitHub https://github.com/libgdx/gdx-video/issues/22#issuecomment-182356468.

[MaximilianWang]

Hello,

I have added the one you provided in #2 before and the project would still run and not really perform any differently than if the jar is not added. Is the only way to really tell if the libraries are being used is if I compile it to do mp4 and it actually plays the file?

Regards,

N

[RBogie]

I don't know of any other way myself, so I guess it's the only way

[MaximilianWang]

Hello,

I just stepped through the library loader stuff using JniGen to see if the .dlls were properly loaded. Now all that is left, i guess, is to figure out why the darn thing won't build for linux64. Maybe I need to dualboot linux instead of doing it through Ubuntu 14 on virtual box. When you built it, which linux distro did you use? Is it because I am not using ubuntu 15 that has access to the latest stuff that I can't build?

Regards,

N

[MaximilianWang]

Hello,

I am currently trying to extend your build scripts to add 2 more builds for armv7 linux systems. Currently trying to follow steps shown in http://lakm.us/logit/2012/11/cross-compiling-ubuntu-arm-ffmpeg/ and adjusting the buildxml to match. Does that link look okay? Sorry for all these questions but the arm7 has me more confused than before.

Regards,

N

[RBogie]

Maybe I need to dualboot linux instead of doing it through Ubuntu 14 on virtual box. When you built it, which linux distro did you use? Is it because I am not using ubuntu 15 that has access to the latest stuff that I can't build?

I don't think that is the issue with building the linux64. I've compiled in a virtual box before, and I've compiled on several distros (Including ubuntu 12 and 14), and all of them compile ffmpeg just fine. What I think that the problem is, is that the patch we apply does not work correctly with the h264 codec in the 64bit build process on linux. You can try if it is indeed the h264 codec by trying to compile the non mp4 version yourself. If that does compile fine, your virtualbox is ok, and the problem is with including the codec.

Does that link look okay? Sorry for all these questions but the arm7 has me more confused than before.

That link looks fine. I do recommend to use --prefix to set a location to install the libs, and then run make install, instead of trying to create a deb file.

[MaximilianWang]

buldfiles.zip

Hello,

So i created a new file build-ffmpeg-arm-linux32.xml file that is able to build fine after i installed the arm-linux-gnueabi-gcc thing from the link and also created a file build-arm-linux32.xml like you have with the other builds but I am not sure what I need to update in this file. I have changed to library destinations and all that to match where i put the compiled code from build-ffmpeg-arm-linux32.xml but I am getting:

../../ffmpeg/libs/arm-linux32/lib/libavcodec.a(ac3tab.o): error adding symbols: File in wrong format

When I run ant -f build-arm-linux32.xml after I first run build-ffmpeg-arm-linux32.xml. Am I totally doing this wrong?

I have attached the build files for your reference.

Regards,

N

[RBogie]

Could you go to this folder, and run file libavcodec.a? Put the output in a response. On Feb 11, 2016 08:21, "NauE5" notifications@github.com wrote:

Hello,

So i created a new file build-ffmpeg-arm-linux32.xml file that is able to build fine after i installed the arm-linux-gnueabi-gcc thing from the link and also created a file build-arm-linux32.xml like you have with the other builds but I am not sure what I need to update in this file. I have changed to library destinations and all that to match where i put the compiled code from build-ffmpeg-arm-linux32.xml but I am getting:

../../ffmpeg/libs/arm-linux32/lib/libavcodec.a(ac3tab.o): error adding symbols: File in wrong format

When I run ant -f build-arm-linux32.xml after I first run build-ffmpeg-arm-linux32.xml. Am I totally doing this wrong?

Regards,

N

— Reply to this email directly or view it on GitHub https://github.com/libgdx/gdx-video/issues/22#issuecomment-182745046.

[MaximilianWang]

Hello,

Thanks for the continued patience. I did a file and the result is here:

root@something-VirtualBox:/soimewhere/gdx-video/gdx-video-desktop/jni/ffmpeg/libs/arm-linux32/lib# file libavcodec.a libavcodec.a: current ar archive

[RBogie]

Whoops, I knew I shouldn't have replied on my phone. This is not the output I actually need. I think you need readelf -h. The info I'm looking for is the platform + abi used in the file

[MaximilianWang]

Hello,

I piped the readelf -h into a file and it has been attached.

In the instructions provided at places like http://lakm.us/logit/2012/11/cross-compiling-ubuntu-arm-ffmpeg/ and https://trac.ffmpeg.org/wiki/CompilationGuide/RaspberryPi that target arm systems, how important is this toolchain thing crosstool-ng? Because I was able to build using the xml file build-ffmpeg-arm-linux32.xml that i made before I finished the steps to set up that. Could that be the cause?

output.txt

Regards,

N

[RBogie]

The ffmpeg compile is going fine. The .a files it is producing are targeted for ARM 32bit, which is what you're trying.

This means that the issue is the build of the jnigen wrapper. The only way this can happen is if you're using the wrong compiler in your build-arm-linux32.xml file. Did you use the correct arm compiler in that file, or are you using the normal gcc (normal gcc will not work with your arm files)?

Also, since the ffmpeg crosscompile seems to go fine, I don't think the crosstool-ng (or lack of) has any negative effect.

[MaximilianWang]

Hello,

I just used the existing build-linux32.xml file as basis and only changed the directory paths to point to where the ffmpeg compile output were. So I need to change the gcc sections in the file to be something else?

Regards,

N

[RBogie]

Yeah, gcc compilers for the native platform, in this case x86. Since you want to compile for arm, you'll have to use an arm compiler. On Feb 11, 2016 22:49, "NauE5" notifications@github.com wrote:

Hello,

I just used the existing build-linux32.xml file as basis and only changed the directory paths to point to where the ffmpeg compile output were. So I need to change the gcc sections in the file to be something else?

Regards,

N

— Reply to this email directly or view it on GitHub https://github.com/libgdx/gdx-video/issues/22#issuecomment-183071967.

[MaximilianWang]

Hello,

So I installed gcc-arm-linux-gnueabi and binutils-arm-linux-gnueabi, do i just update my build-arm-linux32.xml file and find + replace every instance of 'gcc' with 'arm-linux-gnueabi-gcc' without tweaking any of the other flags or variables?

Regards,

D

[MaximilianWang]

Hello,

A bit unrelated but I have been trying out the windows64 build with mp4 support for testing and I have noticed that on full HD videos 3000-6000 bit rate or data rate, the video is quite choppy or jittery. Does it need https://trac.ffmpeg.org/wiki/HWAccelIntro to be incorporated and compiled with the ffmpeg thing? Or is it my flags when I compiled the ffmpeg dll? Or is it on the videoplayerdesktop code side?

Regards,

D

[RBogie]

Hello, So I installed gcc-arm-linux-gnueabi and binutils-arm-linux-gnueabi, do i just update my build-arm-linux32.xml file and find + replace every instance of 'gcc' with 'arm-linux-gnueabi-gcc' without tweaking any of the other flags or variables? Regards, D

Yes, you'll have to replace all instances.

A bit unrelated but I have been trying out the windows64 build with mp4 support for testing and I have noticed that on full HD videos 3000-6000 bit rate or data rate, the video is quite choppy or jittery. Does it need https://trac.ffmpeg.org/wiki/HWAccelIntro to be incorporated and compiled with the ffmpeg thing? Or is it my flags when I compiled the ffmpeg dll? Or is it on the videoplayerdesktop code side?

If it starts to stutter, this means that your cpu is not fast enough for ffmpeg to decode that specific file. It can be solved by using hw acceleration, which requires aditional steps during the ffmpeg build. Since I'm no expert on ffmpeg compiles, I can't help you any further with this part.

[MaximilianWang]

Hello,

While I wrestle with the hardware accel stuff (my cpu is only 15-30% used but video is slow) are you able to enlighten me if you will be implementing a volume control and/or seek functionality in this library in the near future?

Regards,

N

[RBogie]

If your CPU has multiple cores, than the cpu might never go over 15-30% while the video is decoding as fast as possible on a single core.

Volume control is really doable. I'll look into it when I have time. Seek functionality will take quite a bit of work for desktop. It might be implemented, but it's not a short term plan.

[MaximilianWang]

Hello,

What would I roughly need to do to do simple implementation of volume that would work control for android + desktop until I can pull your proper code?

Regards,

N

[RBogie]

Obviously, I would prefer if you would just create a pull request with your code, because the implementation is quite simple.

• You would have to add a method to the videoplayer interface for setting/getting volume info.
• For android you would link this function to Android MediaPlayer.setVolume
• For desktop you would use the RawMusic audio object in the player, which extends OpenALAudio, which gives you control over volume.

[MaximilianWang]

Hello,

Pull request created. I have noticed a slight difference in behavior between the android player and the desktop one during testing. When play(file) is called, android will start the audio of the video even if the render() is not called whereas in the desktop version the audio will not start until render starts. Is this intended?

Regards,

N

[RBogie]

This behaviour is not completely intentional, but it is something I encountered during development. The issue is that develop needs to have a call on which to render a frame. Android however does not. It will simply start playing in the background, and whenever a frame is rendered with a certain texture, it has to most up to date video frame in that texture. This means that syncing the audio on android is something that is done automatically, while on desktop the audio has to be synced to the video frames manually.

To do this syncing correctly, the audio is bound to the rendering of the frames, which also means that before rendering the first frame, there will be no audio.

[MaximilianWang]

Hello,

I see. I have been trying different devices and I am currently experiencing rendering issues on some Android mini PC devices like https://www.giadapc.com.au/shop/q30-fanless-mini-pc. When I run the app the screen generally remains black or white but the audio works fine. If I break point throughout the render function within VideoPlayerAndroid and I occasionally see a glitched out render of the current video frame. Is this because we trying to draw the video frames onto a texture in gdx rather than a native android UI component that we can get occurances like this?

Also I have having trouble positioning this video player around on a stage, it just sticks to the bottom left corner. Playing video seems to also disable any of the other draw texture stuff I am doing elsewhere unless I place them in another stage and render it after the stage that has the media player. What am I doing wrong?

Regards,

N

[RBogie]

Some cheap devices don't fully support the functionality in GPU drivers that google states as supported. Those devices only put a videoplayer app on the phone, of which the activity is used for playing files. However, textures in opengl are not supported in this case. There's not much we can do about this.

The positioning depends on which method of playing the video you're using. If you're manually supplying a mesh, there are some tasks you'll have to do yourself. The documentation should tell you more about this.

[MaximilianWang]

Hello,

I see, that is a shame. I worked around it by extending the VideoPlayerAndroid class and instead have it draw the video on a surfaceview and have it play on top of the view that contains the libgdx screens.

I still don't know why playing a video that is resized to only fit a corner of the screen while on desktop end up causing any other texture rendering stuff i am doing for other actors elsewhere to not appear at all. Is the video player on desktop redrawing the entire page for each frame thus counteracting the draw methods of the other actors? If i disable the video player on desktop, the other actors draw no problem.

Regards,

N

[RBogie]

I still don't know why playing a video that is resized to only fit a corner of the screen while on desktop end up causing any other texture rendering stuff i am doing for other actors elsewhere to not appear at all. Is the video player on desktop redrawing the entire page for each frame thus counteracting the draw methods of the other actors? If i disable the video player on desktop, the other actors draw no problem.

I don't know why this is happening. If you happen to find out why this happens, and it is a bug in the framework, I would love to see a PR with a fix. However, right now I can't help you with it.

Since this issue thread is growing and growing, and is not specific for a single issue anymore, I'm going to close it. If you ever find a new issue, please create a new one, together with some example code on how to reproduce it (As small as possible). Also state the expected outcome and what actually is happening instead.