Support static linking of sfeMovie

[Ceylo]

AC:

• I want to be able to statically link sfeMovie, even if it makes my program LGPLed
• This must work on every supported OS and compiler

[bluekirby0]

Main thing to consider is the runtime libraries used for all libraries need to be ABI-compatible (preferably identical). This is naturally possible under *nix systems without much headache as you just use the same compiler to do everything (and the same runtime library will be linked, barring developer insanity by using options they have no business touching).

Under windows, however, this could only really be supported with either builds done entirely in mingw or builds done in msvc 2013.

Anyhow, I'd be curious what other problems you might run across preventing static builds being possible.

[Ceylo]

Why specifically the 2013 version of msvc? I don't exactly know what runtime SFML uses but… if SFML and sfeMovie are built as static libs, isn't the only linked runtime the one chosen by the final application? In this case I suppose there is no possible runtime conflict.

Arg, except if the runtime is statically linked into one of the static libraries? I don't know if SFML does that…

[bluekirby0]

MSVC 2013 would be required if using MSVC at all because it is the first version that supports C99 (which is used heavily in FFMpeg, your biggest supporting library).

As for runtime libraries, msvcrt versions are not necessarily ABI-compatible. Supposedly versions from v9-v12 ARE ABI-compatible, but for versions that are not, what this means is even if something built against one runtime library is eventually linked against another, the calling conventions used to call core library functions can actually be wrong which leads to ridiculously problematic situations to debug.

Even more likely to break are STL templates, because prior to VC9 the internals of those were almost entirely different from how they are handled now, and STL stuff is already prone to issues when being passed across execution boundaries.

[Ceylo]

For VS2013 it depends on whether the conversion script is used. If it is, it could be built with earlier versions (do we even want to support these for the next version of sfeMovie?).

As for VS versions, I don't want to provide support for version below VS2010, so the VC9 issues do not apply in this case. But anyway yes the user will have to use the same compiler version, that's what SFML already supports by providing binaries for each version. As for which runtime library is used by SFML, it'll need to be checked so that sfeMovie uses the same to avoid issues.

This will be done before this branch is merged back to master.

[bluekirby0]

The whole reason I'm against supporting versions earlier than 2013 for this project is because the reason for the problems and added hassles is MSVC not being standards-compliant for a 14-year-old accepted standard library extension until that version.

I personally think it is crazy to jump through hoops to support compilers that don't conform to the standards required by all of your support libraries, but if you can come up with a way to make it work without using MSYS or cmd directly then I'll cheer for you. CMake is quite flexible so its entirely possible to find a solution here that will let you support older versions without adding to developer headaches.

EDIT: Ahh I forgot about the build system issues...building without bash is practically impossible but you can still skip some steps in the setup by using 2013.

[Ceylo]

Unfortunately FFmpeg can't be built with CMake so I'm afraid MSYS will remain a requirement for a long time... And with the current build system VS2013 or earlier doesn't really matter.

But yep I agree with you VS is a pain with standards

[bluekirby0]

I'm working on a simple minimal MSYS environment (without a mingw toolchain) you could call from an MSVC build environment just to build the ffmpeg msvc version. When I've got it working with vc12 you can tweak it to add in all the extra steps for vc10/11 compilers if you wish.

[Ceylo]

It looks interesting but note that its integration will depend on the needed work to keep the build system working :) As current solution works fine I won't change unless it brings noticeable advantages.

[bluekirby0]

Your current solution relies on building a specific ffmpeg version you provide ahead of time, and building it in a custom location instead of the cmake build folder. What I am doing is setting ffmpeg as a git submodule and providing a minimal MSYS that can work with any CMake generator on windows without having to set up custom things that operate outside of the builds folder given to CMake.

You can be guaranteed to have an up-to-date ffmpeg library this way, and I've already torn out several lines from your existing build system that make (possibly false) assumptions about the build environment. Once I have it all cleaned up and ensured its working on MSVC/MinGW/Debian i'll put it up on my fork with a pull request and you can see what I've done with it.

I've also tweaked a few of the ffmpeg-specific build options to better suit windows builds (i.e. changed from pthreads to w32threads and enabled runtime cpu detection on windows so builds on one cpu won't force the use of code incompatible with another).

EDIT: I would also suggest you treat yasm as something that should be expected to exist in the build environment already. It is incredibly easy to install on any system, and compiling it yourself will horribly break cross-compiling unless you take that into special consideration in your build system.

[Ceylo]

The build in a custom location is to avoid issues with build paths containing spaces. This is not correctly handled by FFmpeg. So if you download FFmpeg into the source's directory, you'll encounter the same issue. It's not related to MinGW or MSYS, only FFmpeg.

As for pthreads, do you know if they work slower than w32threads on Windows? If not, why change this? As for runtime cpu detection, the option with FFmpeg 2.2.2 is "--disable-runtime-cpudetect" so I suppose the default is enabled, isn't it? or did it change in the meantime?

As for cross-compiling and yasm, no I didn't take this into consideration, but considering sfeMovie's development state this is not a priority. I prefer not to bother with this for now.

[bluekirby0]

I'll look into issues with spaces in path names...there is probably a better solution to that problem. In my case, my C drive on windows (SSD) has only 1-2 GB of free space so build stuff cluttering it up can have a pretty negative impact on my system. I also don't have a D E or F drive...my other drives are assigned to J and Q (intentionally put where drive lettering conflicts would not be a concern with the addition and removal of temporary devices).

pthreads-win32 which is the pthread library used by mingw is basically a wrapper around native threading intended to provide windows threading support to programs written specifically for pthreads. It adds some overhead that just isn't there on windows when using native threading. pthreads are perfect for POSIX environments that support them but since the ffmpeg crew bothers to maintain native threading for windows, its best to take advantage of that.

Last time I worked heavily with ffmpeg the default was to disable runtime cpu detection because the general assumption was that the library was being built on the system it would run on, or the target architecture would be set to the lowest-common-denominator (as it would be in a distro build). The option bloats the size of the resulting library because when it sets up the initial function offset tables at start it has to have separate implementations of the functions to point to based on what assembly is supported, but with windows the advantages far outweigh the disadvantages.

I will advise you now, if you have never worked on a large project before you will find it will only get harder to make changes to the basic stuff as it grows and gains complexity. If you don't consider the possibility of cross-compiling now it will be a huge headache once people start trying to build your library to use in their Android or iOS apps.

[Ceylo]

I understand putting files in the C drive can be an issue. As I don't think FFmpeg build system can be fixed easily, the best way is probably to allow the user to define this directory-without-special-chars-and-spaces-in-its-path.

As for pthreads, I had chosen it to fix a build issue, see commit 702017873b7a5513697e02d13a622b5690b56617. I'm not saying it is the best fix but I had changed Windows option on purpose. Maybe this has been fixed in the meantime with later versions of FFmpeg.

As for cpu detection, that's also what I remember, so it seems like they want to go for product portability now.

As for sfeMovie being a large project, this is optimistic :) I'm not saying it'll never be, but if it becomes so, it won't be thanks to me: I have limited free time and I can't make it as big as you seem to think it could become. Considering that keeping the build system working with desktop computers is still not perfectly mastered, cross-compilation is too high a step for now. So whether sfeMovie can go up to this mostly depends on people like you.

[bluekirby0]

I might need to replace your bootstrap shell script with a small C program to fix the issue with spaces (windows cmd shell is REALLY limited in how it handles some things when you are used to bash and zsh), but it should be possible. Other than the issue with spaces I have the ffmpeg builds working with a small msys shell now, with the builds going into the cmake builds directory. I kinda want to get it fully working before I push things to my fork but if I can't finish up today i'll at least put it up for the sake of tracking progress on it.

I guess my next task then will be trying to duplicate your issue with w32threads...is there an old bug report with more details on that? I'd rather fix native threading on windows than rely on a posix threading wrapper. I can say as of the current FFMPEG revision w32threads builds and links fine with sfeMovie, but I've not tried to build the unit tests yet (my boost libraries are probably 14 months out-of-date at this point because most of the stuff I used to use from it are part of C++11 now).

I'll double-check that runtime cpu detection is the default...if not then we can pop the option in on windows builds at a minimum.

Its hard to say right now how big this project will get, but it fills in a huge gap in desirable SFML functionality...and for us old hands at C++ who love the language with a passion, SFML is a gorgeous piece of work and a huge time-saver when it comes to "write once, run everywhere" programs. Once I have this working to the extent I need and the Android/iOS stuff is considered stable, I'll be in good shape to work on my own projects.

With the market in its current shape, I see a great need for any SFML libraries to support those two platforms in order to become widely used and accepted. Desktop hardware progress has hit one of its lowest points in decades, while mobile devices are improving by leaps and bounds every few months.

Also consider that many developers use a mingw cross-compiler to support Windows at all. They may not even have a Windows system to run a native compiler on.

[Ceylo]

Note that the issue with special characters is not only with spaces, it's with all characters that have special meaning like parenthesis for example.

I don't know what you plan with the C program but if it removes windows cmd limitations I'm for it.

As for your fork, please separate your work on different tasks on different branches to ease pull requests, it'll also give a better overview of what would be integrated :)

As for w32threads, from what I remember is was a warning or error when running FFmpeg's configure script. So if you have no issue it should be ok.

As for cross compilation, it's true I have not considered mobile OSs as the main target for future development. I understand why it is interesting but I'm just not able to add this support for now.

[bluekirby0]

FFmpeg does have issues in its own build system with spaces in the path when doing out-of-tree builds, but I've found (what is in my opinion) a better workaround by using symlinks to /tmp (works even in the msys environment).

The C program basically combines the functionality of your BatchBridgeToShell.bat and RunShellCommand.sh rendering both unnecessary, while also ensuring all arguments are individually quoted so they can be parsed correctly by bash. If we run into any particular situations it can't handle, its fairly easy to extend to handle new cases, and its written in straight C89 so it can be built with the platform SDK (version 6 or higher...I'm using 7.1 targeted at XP) for maximum portability without the need to install special runtime libraries for it. It can also be built easily with gcc, but it produces a much larger binary.

An example of what it outputs:

bash -c '"./build_ffmpeg.sh" "windows" "vs" "1" "Q:/_Projects/path with spaces/sfeMovie-master" "Q:/_Projects/path with spaces/sfeMovie-master-build" "theora" "flac" "vorbis" "opus" "vp8""vp9" "adpcm_4xm" "adpcm_adx" "adpcm_afc" "adpcm_ct" "adpcm_dtk" "adpcm_ea" "adpcm_ea_maxis_xa" "adpcm_ea_r1" "adpcm_ea_r2" "adpcm_ea_r3" "adpcm_ea_xas" "adpcm_g722" "adpcm_g726" "adpcm_g726le" "adpcm_ima_amv" "adpcm_ima_apc" "adpcm_ima_dk3" "adpcm_ima_dk4" "adpcm_ima_ea_eacs" "adpcm_ima_ea_sead" "adpcm_ima_iss" "adpcm_ima_oki" "adpcm_ima_qt" "adpcm_ima_rad" "adpcm_ima_smjpeg" "adpcm_ima_wav" "adpcm_ima_ws" "adpcm_ms" "adpcm_sbpro_2" "adpcm_sbpro_3" "adpcm_sbpro_4" "adpcm_swf" "adpcm_thp" "adpcm_xa" "adpcm_yamaha" "interplay_dpcm" "pcm_alaw" "pcm_bluray" "pcm_dvd" "pcm_f32be" "pcm_f32le" "pcm_f64be" "pcm_f64le" "pcm_lxf" "pcm_mulaw" "pcm_s16be" "pcm_s16be_planar" "pcm_s16le" "pcm_s16le_planar" "pcm_s24be" "pcm_s24daud" "pcm_s24le" "pcm_s24le_planar" "pcm_s32be" "pcm_s32le" "pcm_s32le_planar" "pcm_s8" "pcm_s8_planar" "pcm_u16be" "pcm_u16le" "pcm_u24be" "pcm_u24le" "pcm_u32be" "pcm_u32le" "pcm_u8" "pcm_zork" "roq_dpcm" "sol_dpcm" "xan_dpcm"'

I'll generally only push to my fork when i'm done with a major task...I might make several commits locally but i can wait to push them until you've reviewed the previous pull request. I don't anticipate many of the tasks I work on taking less than a week to finish with my constricted time schedule and the current state of the library.

I'm getting no warnings or feedback particular to w32threads during configure. I haven't sifted through the myriad of warnings that crop up during the make stage, but that has always been an issue with ffmpeg and happens no matter what platform you build for.

Most of what I'll be working on first will be foundation for supporting a cross-compiler-friendly build system. CMake should make the necessary effort minimal while providing the maximum flexibility to support fringe cases as well.

[Ceylo]

Does the symbolic link work fine when linking to a different drive?

As for the generated command, how does it behave if you have parenthesis in paths?

As for w32threads then it's ok, you don't even have to test different build systems as they all rely on MinGW for now to build FFmpeg.

And generally speaking thanks for your help on the project :)

[bluekirby0]

Yes, symbolic linking will allow you to connect one path to another regardless of the physical properties and mounting options involved. By using /tmp in MSYS you are actually using Windows %TMP% (which for most users with limited space on their OS drive will be overridden to point somewhere with more space available).

If we were working with a batch file, parentheses would be an issue and we would want to use ! instead of % to delay expansion of the environmental variables until the next process receives them, but when working from a C program you do not have the same issues with how paths are parsed (though you DO have to take C preprocessor-specific things into consideration, like escaping backslashes in string literals). I will, nonetheless, add parentheses to my test path to make certain.

My current method of building FFMPEG uses MSYS but NOT MinGW. This way I can use the configure script and GNU make but the toolchain used is actually that provided by the version of MSVC you are running the solution generated by CMake in (so it calls cl.exe and link.exe rather than gcc/g++/ld). Right now it only works with MSVC 2013 because I did not add in the C99 wrapper and custom inttypes header, but once I'm satisfied it works properly it should be relatively simple to modify it for 2010/2012 versions.

No problem on the help front. I have some plans for this once it is more or less feature-complete and portable. It is the one feature I would really like a simple, portable solution for that SFML lacks. I'm not a fan of FFmpeg's API (its hard to find good, reliable documentation on it because it keeps changing) but there is no denying its a solid solution for the job. The only real competition (not getting into the whole ffmpeg/libav debate) is GStreamer, and that might be a little heavy for use on mobile devices since it pretty-much works like DirectShow on Windows internally.

[Ceylo]

Ah yes the fact that batch script interpret parenthesis would explain why I was not able to have correct behavior by modifying the shell script.

This great news for VS2013! Do you mean it's already working on your side ? :) If support for older versions of VS is too painful don't bother with it, VS2013 exists for some time now and users can update to use at least the express version. I don't want to pollute the next version of sfeMovie with stuff that will be more or less already outdated. So either it can be done cleanly or don't do it. I have supported backward compatibility for too much time, better have cleaned and easily maintenance code now.

Can you explain in what way GStreamer working like DirectShow makes it heavy? I didn't use both API so I'm interested :)

[bluekirby0]

Yes, I have FFmpeg building under MSVC 2013 right now! Support for earlier versions will require some additionl support tools but should be possible to integrate without polluting the entire build system if desired.

Basically Gstreamer/DShow provide primarily high-level interfaces in which the possible filter chains to produce an output are all calculated and then weighted based on filter preferences. It is a great system for "just works" playback but it tends to do a lot of extra work internally that is somewhat unnecessary when working in a more limited environment.

It is possible to use either with lower-level functions, but it can become somewhat convoluted quickly, and there isn't really a good way to mix high-level and low-level APIs.

[Ceylo]

It is really good news about VS2013!! As for earlier versions, the reason I prefer to avoid too much backward support is to lessen required tests on the build system and spend more time on features.

Ok I understand about GStreamer. I have never tested it so I can't tell, the only point I know is a SFML user had tried to use it to create its own movie player. He had setup everything very quickly and it was working mostly fine. He didn't seem to complain on performance either. He had random crashes he didn't know the source though.

[bluekirby0]

Glad to have it working under 2013. Since its a C library there are a lot fewer linking concerns than working with C++ libs, but it should be possible to pack the entire thing together as a monolithic static library should a user choose to do so (just watch those licenses).

GStreamer is a great bit of software and very stable (as long as you stick to the non-experimental codecs). It is something of a resource hog, though...which is no problem on a desktop system where you generally can't find a decent one with less than 4GB of RAM but for mobile platforms it can start to feel a little bloated.

One other thing GStreamer is missing (or was the last time I used it) is hardware accelerated decoding support. This is crucial on mobile devices as pure software decoders for h.264/vp8/h.265/vp9 are murder on CPUs.

[Ceylo]

GStreamer has this page about hardware decoding so I suppose they have support for it now? The main issue I can see is that they rely a lot on third party proprietary plugins.

http://docs.gstreamer.com/display/GstSDK/Playback+tutorial+8%3A+Hardware-accelerated+video+decoding

Edit: on the other side, FFmpeg requires non negligible work to take advantage of the hwaccels, to avoid having to download the data from the GPU to the CPU memory.

[Ceylo]

Do you have any news about the build system? I'm actually really interested but you don't look to have any commit about it on your fork yet. If there is work left, is it possible to help in order to get this merged as soon as possible?

[bluekirby0]

Oh, right...I got the MSVC stuff working solidly but I probably horribly broke build everywhere else in the process. I keep meaning to go back and try to at minimum get a MinGW and a Debian build working but I need to have enough free time all at once to be able to invest into such an endeavor.

I will go ahead and commit/push what I have with a nice big warning in the commit message about possible issues. Some things have changed that you might not expect or agree with as well so feel free to tweak to your liking...it would be difficult at this point to divide those out into a separate changeset.

[Ceylo]

Wow indeed these are big changes!

I'm globally okay with most of your changes. And you made me think the dependency policy should be better defined (yes, that's the monster below!). You did a good move by removing the dependency to MinGW. That's one less thing the user has to care about. Which leaves us with FFmpeg and YASM.

We have 4 possible choices about dependencies:

• let the user provide all dependencies, including FFmpeg binaries (trickier for the user, but really clean sfeMovie repo)
• pack all the dependencies right into the sfeMovie repo (big repo, heavy manual updates, no additional user download)
• include all the dependencies with submodules (clean repo, lightweight manual updates, Internet dependent, not compatible with GitHub archive provider for direct zip download)
• mix previous solutions (inconsistencies!)

The biggest problem in the past was ease of compilation (and this is crucial as I can't provide sfeMovie binaries with patent-covered decoders), so choice 1 is out.

I would prefer to avoid choice 4 to keep consistency. Choice 2 will make the repository huge. In the past I also included SFML in the repo. But it has been pointed out that users who want to use sfeMovie already have SFML, so relying on the user's SFML installation is ok. Thanks to you Visual Studio users don't need MinGW anymore, and we can of course consider that MinGW users already have MinGW installed. So no problem on that side.

This leaves FFmpeg and YASM. You've chosen to remove FFmpeg and put it as a submodule. Requiring the user to download FFmpeg himself and give its path in CMake could also work, but then the FFmpeg version being used wouldn't be controlled anymore (is that a problem?). sfeMovie build scripts would still be used to produce FFmpeg binaries, as it's not an easy task.

To my mind this is the most annoying point: if building FFmpeg correctly was easy, sfeMovie's build system wouldn't have to take care of building it. It could just ask the user to tell CMake where it's installed and FindFFmpeg.cmake would do the necessary work. Here a partial workaround can be relying on Zeranoe builds, or even better: setting up a Zeranoe-like build system that creates the binaries for both Linux, OS X and Windows, with versions including either free or patent-covered decoders. But that's work, and it needs care due to the patents.

I checked against FFmpeg and libav website. FFmpeg website provides easier access to binaries for all platforms, but the directly available binaries are GPL3 (Zeranoe) or static FFmpeg executable without the libraries (ffmpegmac.net).

As for binaries provided by the sfeMovie website, this is technically feasible: at the moment I'm setting up a continuous integration system through Jenkins. I use the website's server for Linux builds, a free Amazon EC2 instance for Windows and my own computer for OS X builds. So it could very well be used to build and provide FFmpeg and/or sfeMovie binaries.

As for YASM, it's much much easier to get binaries of it. Although a "standard" SFML user won't already have it installed. So that's an "easy" dependency. The main question I have about YASM is: why are you against building it? is it only because it's not part of sfeMovie and it's easy to install?

[Ceylo]

These discussions make me think there should be a forum for sfeMovie…

[bluekirby0]

I wish I could reply to this properly right now but I will be busy for the next few hours. I will go ahead and say, though, that you are starting to think in the right direction. Submodules are overall much cleaner and can be locked to commit versions if desired, and providing ffmpeg binaries can make the entire process much easier for the end user if you have the resources for it. I still think there should at minimum be a link to the ffmpeg page on how to build it for power users or people who wish to license individual codecs etc...

As for yasm even though it is quick to build in comparison, I equate it to compiling GCC to compile one program. It is probably just personal preference but I do not believe a compiler should be built as part of a build process...especially if it causes that compiler to be built every time.

[bluekirby0]

Ok so...

FFMpeg: It is infamous for API changes and mass deprecation of functions, BUT 1) This infamy comes mostly from the ffmpeg/libav split period where the two competing libraries were trying to keep up/one-up each other and the changes were rather rapid 2) Deprecated functions are always left in for a while to give devs a transition period in which to update their code (stuff builds but spits out warnings)

If you want to have users build it themselves then you will want to add a number of presets to the CMake project so that users can have "common supported builds" that can be easily replicated...i.e. a "default" which only includes royalty-free codecs with LGPL/MIT/BSD licensed code. Then add in presets for codecs that are patent-laden but have LGPL/MIT/BSD licensed code for regions where patents are non-enforceable. A variant for each with GPL-licensed code would be useful for FOSS projects.

If you want to go the whole prebuilt route, make sure that your distribution center is located in France or another country where patents are non-enforceable. IANAL but that should at least provide some protection from patent groups trying to charge per-download fees for your distributions. I can set up build systems for such services if you need any help in that area. Just remember that even if you do this, someone will want some configuration you do not already provide so you will still end up supporting the odd user building their own FFMpeg unless you want to make it possible for people to set up their own build-bot tasks on your server.

The other thing I have touched on with YASM before (and did not in my previous post) is the necessity of supporting cross-compilers somewhere down the line. Requiring YASM to be in the build environment already just simplifies that scenario a LOT (since a win32 YASM can assemble ARM code if tasked to do so, and vice-versa). Even if cross-compiling is not a major concern of yours, it is definitely a concern of mine.

[Ceylo]

Ok, so we would have "Free", "All but GPL" and "All". At the moment Jenkins is already setup to build "Free" and "All but GPL", but with the current build system (so not independent from sfeMovie lib build).

The server for yalir.org is indeed in France so it should be fine. I've sent an e-mail to the Zeranoe builds maintainer to know if he's had any trouble with patent holders. When I had asked him the same 2 years ago, he had said "no".

When you talk about help on the build system, do you mean providing hosting for the binaries, or slave builders for Jenkins? If it's the latest I'd be interested because Windows has more configurations to build as it must support both MinGW and VS2013. At the moment I'm running these builds on a free Amazon EC2 VM, which is a 30GB drive (3GB free left at the moment :smiley: ) and 1GB RAM. If you have experience with Jenkins this is also welcome (can't get builds to be automatically triggered after a push at the moment for example, so I've planned a regular build every evening GMT+1).

As for users wanting custom builds, I won't provide the binaries: if they are at need of such customization, they can spend the time to do the build, eventually by copying configure commands used by standard builds.

As for YASM, I don't know about cross-compilation constraints so I believe you. As for the fact that it's weird to rebuild YASM to build a single binary, I agree too. So ok for this change, although it introduces one constraint: YASM must be reachable from the PATH, or its path WITH NO SPACE must be given to CMake.

[bluekirby0]

I am not so wealthy as to be able to afford a long term commitment to hosting something...especially if it is a windows box as those get VERY expensive if you follow all of their licensing rules. I just meant that I can do the legwork to get things running smoothly for whatever boxes are available. It might be possible to set up a wine nmake monstrosity for automated windows builds but I have never tried such a thing before.

For extra space on your amazon box consider setting up a samba share on another server.

MinGW builds for windows targets can of course be done straight from any Linux environment once cross compiling is working.

I do not have experience specifically with Jenkins but I do have a way of bludgeoning thing into working eventually.

Check for yasm first, then try falling back on nasm. Most sane build environments will have one or the other in the path if they have them at all. I provided a win32 binary for the MSYS build environment in my fork which is the scenario most likely to be problematic for end users.

I am really wishing the ffmpeg guys would fix the no space in paths thing on their end so we do not have to work around it so much.

[Ceylo]

What do you mean with "VERY expensive if you follow all of their licensing rules"? I was rather thinking about whether you'd have a server sleeping most of the time or a spare working computer that you don't use anymore. Or if you could create a free Windows instance through Amazon EC2. But don't worry if you don't want to bother with any of these, the current CI setup is usable :)

As for help, I don't know how motivated you are, but I think it would be better to first finish what you've started with the new build system. However for smoother integration your work would need to be separated in distinct branches/pull requests:

• one for the support of the 2 new decoders
• one for dropping the integrated build of YASM (rely on executable in PATH or give ability to specify where it is, most probably through a FindYASM cmake script, allowing use of nasm as an alternative [is NASM a yasm-compatible assembler?])
• one for the native build of FFmpeg, without breaking build with other compilers. For this point you can already check MinGW, and for the other platforms Jenkins will do the job.

As for cross-compiling, that could do it for MinGW, although I prefer a real Windows system to immediately see Windows specific issues.

[bluekirby0]

Historically windows server licenses have been very expensive, recurring costs, and operating a non server version of windows as a server is against the license. If you are fine with a spare home system on a mediocre internet connection running as a build bot then I can scrape something together.

I stuck everything in the same change set because the codec change affects one line in the build scripts which will probably conflict with other changes to the build script. The integrated yasm build is also something that will conflict with changes to the build script. The only sane alternative is to tackle each task one at a time to avoid merge conflicts.

For fixing other compilers I really need to get it working on my end first so I can make quick changes until everything is settled again. Jenkins is better for testing code changes than for testing build system stuff I think.

Yasm is a nasm compatible assembler that is more portable. Ffmpeg prefers yasm simply because it runs everywhere but nasm will produce the same results if it is available on the platform.

Unit tests for mingw builds might function under wine but the tests will need to be tweaked to check os and use wine if available for Linux mingw builds.

[Ceylo]

Oh thanks for the info, I had never looked at Windows server licences. Considering this I'm surprised Amazon can provide Windows servers for free... What is the speed of your internet connection? I have setup Jenkins to do shallow clone of the git repo, this reduces the download from 100MB to around 20MB. This should help :)

As for branches, if you're afraid of conflicts, just create your branches from the previous ones, and create the pull requests in the same order. It'll still be better to integrate your changes in a controlled way. I won't merge everything as a whole. As for the codecs, considering how small the change is, I agree with putting it with the YASM change. To avoid conflicts with upstream master you should also merge from upstream to your fork first.

Why wouldn't Jenkins be appropriate to check the build system?

Ok for NASM. As for tweaking the tests to use Wine, I'm not sure I want to go that way, looks like introducing hacks to get things done and I prefer to avoid that. We'll see how saturated are the Windows slaves.

[bluekirby0]

Upload speed realistically sits at around 1.5 Mbps. Download speed won't be a problem.

I can just rip out the changes for yasm and codecs for now. I only pushed my current work because you were interested and I did not have a specific timeline of when I could finish.

On my own systems I can get feedback on build system changes almost instantly. Running each minor change through Jenkins can get a little tedious after a while.

[Ceylo]

This is not what I would call a mediocre internet connection haha :D Your upload speed is my download speed :) Plus considering that there will be very little to upload (zipped binaries are small, once artifact deployment is set up..), you don't need to worry. Do you want me to add you as Jenkins admin to play with it and set your computer as slave? I'll be away starting from today so probably not very reactive.

And yes Jenkins is of course to get feedback for systems you don't run, and also with configurations you may forget to test (debug/release, free/non-GPL codecs etc). This lets everyone check that it's ok before merging.

By the way do you prefer to work independently on your fork or as part of the Yalir team with me and feliwir ? Being in Yalir would ease full checks with Jenkins but do as you prefer.=

[bluekirby0]

It is still residential class internet rather than commercial which can have more than double the upload. I guess it is all perspective. I can play with Jenkins and see how sane it is to set up on my end. It would be helpful for checking Mac builds at least since I do not own a Mac.

It does not particularly matter whether I work in a fork or independently. Pull requests are fine and present a layer in which you can provide commentary on contributions before they are merged and I still get credit for them.

[Ceylo]

Ok, so I let your register on https://yalir.org:8080/ I'll set your account to have all permissions once you've created it.

As for the Mac slave, you'll notice it's not always connected. It depends on whether my Internet connection is on (free hotspot which requires reconnecting every hour usually, except for the next few days where I'll have a normal connection). So Mac results won't be immediate but I've not better solution for now. Don't hesitate to queue several builds if you want to and if they're only delayed because of the Mac slave.

Also for your fork to be tested you'll need to add one more Git source in Jenkins.

[Ceylo]

This is very similar to issue #48 where the conclusion was to bring support for static linking on Windows only. It's indeed not really meaningful on other platforms. So unless someone comes with good reasons for Linux or OS X I consider this closed :)