search

desura / desura-app

Free online games platform, with an open source client. In these scenarios, Desura's free online games are the perfect solution. These games don't require a high-powered computer or lengthy installation processes.
https://www.desura.com/
Other
121 stars 31 forks source link

[CLOSED] Simplifying the build system #16

Closed lodle closed 10 years ago

lodle commented 10 years ago

Issue by naryl from Tuesday Nov 22, 2011 at 00:36 GMT Originally opened as https://github.com/desura/Desurium/issues/16

The current build system which produces a distribution which should work everywhere is actually only useful for a rare and special case when the user downloads it from the Desura web site. I suggest to reserve it for the project maintainer (Lodle) so that

  1. Other developers and package maintainers won't have to deal with occasionally forgotten or outdated build scripts
  2. Lodle won't have to spend time maintaining the scripts so they would work for everyone when they do a lot of extra actions not required for either contributors or package maintainers.

Instead we should implement a simple build system which should only use system libraries which would simplify work of contributors and package maintainers (who would currently have to clean up all the bundled libs and fix scripts which build them).

lodle commented 10 years ago

Comment by Jookia from Tuesday Nov 22, 2011 at 00:49 GMT

The amount of boilerplate code in src/third_party/build_lin/Build*.sh is terrifying, and on top of that there's two shell script systems, one for Windows, one for Linux.

I don't know of any cross platform shell scripting languages, or even those that allow functions and files to be included. Maybe Lua would be useful in this case?

lodle commented 10 years ago

Comment by naryl from Tuesday Nov 22, 2011 at 01:06 GMT

Perl is commonly used for crossplatform build scripts. And Tcl is worth considering too for its bash-like syntax, especially in the EXEC command.

lodle commented 10 years ago

Comment by eXeC64 from Saturday Jan 21, 2012 at 16:56 GMT

CMake is quite a popular cross-platform build system. It just generates native Makefiles or Visual Studio projects, or even XCode projects. Might be worth using that. The building could be simplified down to a couple of CMakeLists.txt files.

lodle commented 10 years ago

Comment by Jookia from Sunday Jan 22, 2012 at 02:30 GMT

Possibly, but we still have to pull in third party applications.

lodle commented 10 years ago

Comment by yurikoles from Sunday Jan 22, 2012 at 20:11 GMT

No, that all magic of CMake and premake. They are searching for required dependencies and generates resulting Makefile. User just have to install missing dependencies in system's specific way.

lodle commented 10 years ago

Comment by krnekit from Sunday Jan 22, 2012 at 20:33 GMT

CMake has ExternalProject module, that can download sources or fetch them from CVS/SVN/Git, so fetching and building external stuff is possible.

lodle commented 10 years ago

Comment by miloongithub from Tuesday Jan 24, 2012 at 16:04 GMT

IMO CMake is best possible option here.

lodle commented 10 years ago

Comment by Jookia from Tuesday Jan 24, 2012 at 16:26 GMT

If any of you want to try moving it to CMake, I'd help out.

lodle commented 10 years ago

Comment by yurikoles from Tuesday Jan 24, 2012 at 18:25 GMT

If we want to push desurium in major Linux distributions we must avoid using "ExternalProject" module. Wee need to use distribution's packages. No more standalone builds. I think there must be poll on sersious discussion on standalone/distro variants. In addition server team must provide stable protocol or API specification and maintain it.

lodle commented 10 years ago

Comment by Jookia from Tuesday Jan 24, 2012 at 18:27 GMT

Stable protocol or API specification? What does this have to do with the build system?

As for using the system's library, I agree, but it's not possible as the third party libraries are patched.

lodle commented 10 years ago

Comment by miloongithub from Tuesday Jan 24, 2012 at 18:35 GMT

yurikoles, most part of GNU's userland build process is based on GNU Build System's (or Makefiles if you like) stuff. And distributions' packagers are taking care of doing RPMs, DEBs and whatever you want from that. And this approach is working. Some people are providing mechanisms for their projects to build distro-oriented packages, but this is not priority nor some best practice.

lodle commented 10 years ago

Comment by yurikoles from Tuesday Jan 24, 2012 at 19:36 GMT

When i say "distro variant" I mean that build system works with all major distribuiton supplied -dev packages.

lodle commented 10 years ago

Comment by yurikoles from Tuesday Jan 24, 2012 at 19:46 GMT

I think patching third-party libraries is wrong way. Desurium is no longer proprietary software or game. There is no need in static builds.

lodle commented 10 years ago

Comment by miloongithub from Tuesday Jan 24, 2012 at 20:02 GMT

yeah agree; that's also part of the plan I hope.

lodle commented 10 years ago

Comment by Jookia from Wednesday Jan 25, 2012 at 06:54 GMT

Of course patching third party libraries is the wrong way to go. But the patches are needed, and nobody's checking with upstream if the problem's been solved.

lodle commented 10 years ago

Comment by nitrix from Wednesday Jan 25, 2012 at 07:44 GMT

I'd recommend CMake's ExternalProject or Git submodules... seeing how many third party libraries this project needs.

lodle commented 10 years ago

Comment by Protektor-Desura from Friday Feb 03, 2012 at 06:37 GMT

If you want to remove all the custom patches to the libraries/etc. used someone is going to have to spend the time to see if the issues have been fixed and if not then submit the patches for inclusion in the standard version. It will take time but I don't see a way around that. Maintaining our own patches for the long term will chew up resources that could be used somewhere else more effectively.

lodle commented 10 years ago

Comment by Jookia from Sunday Feb 05, 2012 at 08:45 GMT

I have to side with Protektor on this. I'm okay with downloading and compiling libraries that the system doesn't have (for instance, curl + c-ares), but the patches are going to cause problems with upstream.

lodle commented 10 years ago

Comment by nitrix from Sunday Feb 05, 2012 at 09:07 GMT

Not necessarily. If the 3rd library is a git repository, you can specify up to which commit you want to clone using submodules.

Then once in a while, you update the submodule from upstream, check if the patch still works for it, if it does, push the submodule update. If it doesn't, fix the patch, then push both the submodule update and the patch for it.

Its the same amount of work, except the library is hosted remotely and updated only when you see fit (so it doesn't break your code, patches or API that rely on it). No space wasted + official 3rd library's bug fixes. That's a good benefit.

TL;DR: Submodules are awesome.

lodle commented 10 years ago

Comment by Protektor-Desura from Monday Feb 27, 2012 at 06:56 GMT

Yes, but you just listed the one thing that will chew up time that could be used elsewhere more effectively...."fix the patch". If we can remove that whole issue by spending the time to get it included upstream then the effort spent on that is better spent for the long term benefits, which should provide labor available for other areas to move the client along faster.

lodle commented 10 years ago

Comment by olavd from Sunday Mar 04, 2012 at 15:48 GMT

Well, it certainly bears the trademarks of a closed source past. All this nastiness could also contribute to the unstable nature of the client(s), but the Desura Server could be just as guilty here.

lodle commented 10 years ago

Comment by Jookia from Sunday Mar 04, 2012 at 23:00 GMT

This is Desurium. We don't care about the server.

lodle commented 10 years ago

Comment by karolherbst from Sunday Mar 04, 2012 at 23:57 GMT

I've begun to make up a cmake based project in a fork. Target system is gentoo linux until everything works fine.

I'll try to use the files in the repository and external dependencies only. At the current state I fight a little bit with symbols in the shared libraries while executing the Desura binary (most of the desurium stuff is building right now).

Can somebody check, if the original build system works with my fork? I don't want to break something at this state. Further can you check, if everything is building right now? Please commit some bugs, so I can take care of this.

EDIT: I noticed that I can't activate a Issue page for my fork, so write me a message or something else.

lodle commented 10 years ago

Comment by Jookia from Monday Mar 05, 2012 at 00:59 GMT

I just looked at your fork, it has a few problems: It doesn't build Curl or WxWidgets or bzip2 or whatnot.

lodle commented 10 years ago

Comment by Jookia from Monday Mar 05, 2012 at 01:23 GMT

Nice work on the CMake branch. I'll see if I can get it to work on my machine.

lodle commented 10 years ago

Comment by olavd from Monday Mar 05, 2012 at 01:39 GMT

Exciting news though.

lodle commented 10 years ago

Comment by Jookia from Monday Mar 05, 2012 at 03:59 GMT

I can't get Breakpad to work with the CMake branch. There seems to be no such file as google_breakpad/exception_handler.h

lodle commented 10 years ago

Comment by karolherbst from Monday Mar 05, 2012 at 09:39 GMT

-- I just looked at your fork, it has a few problems: It doesn't build Curl or WxWidgets or bzip2 or whatnot.

yeah, thats beacause cmake use externel dependencies, so you must have this installed on your machine already (later I will invoke build scripts, if it's not found, so cmake will fetch dependencies itself)

find_package(Boost COMPONENTS date_time filesystem thread REQUIRED) find_package(BZip2 REQUIRED) find_package(CURL REQUIRED) find_package(GoogleBreakpad REQUIRED) find_package(GTK2 REQUIRED) find_package(Sqlite REQUIRED) find_package(wxWidgets 2.9 REQUIRED)

there cmake looks for all the libraries.

-- I can't get Breakpad to work with the CMake branch. There seems to be no such file as google_breakpad/exception_handler.h

have you installed google_breakpad? On my machine I have installed all this libs via portage (google breakpad is installed under /usr/include/google_breakpad).

And can you please post the cmake output via pastebin or whatever, so it will be easier for me to debug this stuff. At this time, I work with somebody else, who has gentoo also and he checks this fork on his machine.

lodle commented 10 years ago

Comment by Jookia from Monday Mar 05, 2012 at 11:48 GMT

Arch Linux has no Google Breakpad package, and finding the compiled version's include folder is nontrivial.

As for using external dependencies, it's not okay to do that at the moment. The third party dependencies are patched or configured properly. The code will break without the patches, which leads me to wonder if you've actually got a working build out of the current CMake system. Also, Curl must be built with c-ares, which most distros don't do.

lodle commented 10 years ago

Comment by karolherbst from Monday Mar 05, 2012 at 11:59 GMT

cmake can patch externel dependencies and check them out via svn ;) (at the moment I do it for wxWidgets)

The next step will be build Google Breakpad if it is not installed