search

Secretchronicles / TSC

An open source two-dimensional platform game.
https://secretchronicles.org/
GNU General Public License v3.0
204 stars 50 forks source link

MinGW/MSYS2 native build support on Windows #224

Open ghost opened 9 years ago

ghost commented 9 years ago

This is something that I've been thinking about a lot lately since I remember SMC used to use Windows almost exclusively for it's development before FluXy left and had everything needed to make compiling easy on Windows. The project has always been easy to compile on Linux due to it's (really amazing) package management system. This issue is mainly geared toward making Windows native compilation easier and more welcoming for attracting Windows developers.

P.S. Off topic but Linux made me realize how nice having a package management system really is... One click updates for everything :)

datahead8888 commented 9 years ago

Yeah...we really should figure out the steps to get it to build in Visual Studio again. This will attract a number of programmers who prefer Windows development.

We will need to outline the steps to use CMake with Visual Studio (and it would be nice if we found a plugin so that people could point and click in Visual Studio and have CMakeLists.txt updated for them).

My copy of Visual Studio is an academic copy from the University, so I may need to double check my licensing before doing much with it. There is also an express copy of Visual Studio out there, but I'm not sure if I can easily run both.

Another issue that will come up is whether people who are developing on Windows can (or should) test Linux building. We're currently verifying that MXE cross-builds work correctly each time we push something to the repository from Linux. People strictly developing from Windows will not be able to do for this for Linux without either installing Linux or using a virtual machine.

ghost commented 9 years ago

I haven't looked for any plugins for Visual Studio regarding CMake yet... The main troubles I've encountered so far is building the dependencies themselves since the pre-built binaries for Windows I've found floating around are extremely out of date.

I have had success before running different versions of the Express editions on one system, however if you're not sure, the Windows 10 Tech Preview is available for free until it expires in May-ish for anyone who wants to test it.

datahead8888 commented 9 years ago

There are some tools on Windows similar to apt-get: https://chocolatey.org/

If you're able to document the dependency steps, I'd be interested in setting it up on my machine at some point in the near future.  I like to use Visual Studio's debugging tools.

Quintus commented 9 years ago

I will look into building SMC natively on Windows with MinGW sometime, but I’m currently waiting for them to officially support GCC 4.9. Once they do, we can use all the nifty features C++11 gives us.

Vale, Quintus

datahead8888 commented 9 years ago

@dlpenguinlover - would you be interested in Visual Studio support in the long term?

I can't look into it now, but if the dependency issues are resolved on Windows it may become feasible.

MinGW would also be good, especially given that it's open source.

ghost commented 9 years ago

I'd be willing to look into it. Right now, I'm trying to figure out how to get CMake to find libraries and includes on Windows... I've gotten it to find rake, gperf, rake, and the other binaries that it requires. I'll mess around with dependencies after I get that worked out.

MinGW I've heard is an amazing environment to work in, I just don't know anything about it. I'll be sure to do some research on that as well.

On Oct 6, 2014, at 6:18 PM, datahead8888 notifications@github.com wrote:

@dlpenguinlover - would you be interested in Visual Studio support in the long term?

I can't look into it now, but if the dependency issues are resolved on Windows it may become feasible.

MinGW would also be good, especially given that it's open source.

— Reply to this email directly or view it on GitHub.

datahead8888 commented 9 years ago

If we can support both Visual Studio AND MinGW...we'll have power at our fingertips... :) :) :) :) :) :) :) :) :)

Bugsbane commented 9 years ago

I know almost nothing of compiling on Windows, but I will say that while I am a big supporter of the idea that extending to Windows / MacOS can give a project more reach and developers, that every single time I've followed a Linux only project extending to native Windows support, they've always been amazed at how horrible the experience was, how many undocumented bugs there are, how horrible the tools are and how much longer it took than expected. If we plan on undertaking this, I would definitely talk this through with Jonathon from OpenShot and/or Boud from Krita. I know they worked through a lot of the hard issues before.

There be dragons here.

datahead8888 commented 9 years ago

As far as I know the old Secret Maryo Chronicles did work in Visual Studio. We would just have to get to a point of having good, documented procedures and check if any code compatibility issues have someone cropped up. Checking with someone from another project is not a bad idea at all, though.

Bugsbane commented 9 years ago

Cool. Sounds good.

Quintus commented 9 years ago

Let me clarify: TSC already runs on Windows. The beta4 has an installer that actually works. The Windows binaries are created by means of crosscompilation from Linux to Windows, where the toolchain (MXE) is based on MinGW.

I’m currently trying to do a native compilation of TSC on Windows with MinGW. It turns out that it is a nearly infinite amount of work, because you have to compile all the dependencies. I’m currently working on smpeg, a dependency of SDL_mixer. After that, SDL_ttf, FreeImage, and CEGUI are still needed (with dependencies), then everything is there. I’m working on that since yesterday morning, and compiling all these stuff takes eternity on Windows. Plus sometimes obscure linking errors occur.

Valete, Quintus

datahead8888 commented 9 years ago

Quintus, it's probably better to focus on the release now than get MinGW working. It would be nice to have support for it, but the release is a really big deal for the project now.

Quintus commented 9 years ago

Quintus, it's probably better to focus on the release now than get MinGW working. It would be nice to have support for it, but the release is a really big deal for the project now.

Probably you are right. Nevertheless, I would like to post my progress. I have successfully built ALL of TSC’s dependencies with MinGW. Read that again. It took two days. I also successfully compiled TSC, but it failed to link with thousands of linking errors I am too exhausted to fix. During the attempt I wrote a file where I noted all the steps I took to reach this state. The file is available via Gist if you want to give it a shot also.

If anyone of you feels like dealing with missing -static switches or wrong order of libraries in the linking commandline (Windows GCC is very picky about that one), then just try. Any corrections to CMakeLists.txt are greatly appreciated (and you will have to edit that file heavily for a native Windows build I fear).

Some notes:

For the next time, I will restrict my Windows TSC experience to crosscompiling TSC, which works just fine.

Valete, Quintus

datahead8888 commented 9 years ago

Probably you are right. Nevertheless, I would like to post my progress.

I just figured you were doing it for us. I don't want to otherwise discourage you. The release, though, is the top priority for giving something to everyone. Thanks for looking into this.

The reason I'm interested in Visual Studio is because of my familiarity with the IDE and because I really like the debugging system. I would be interested in eventually trying MinGW out, especially after @dlpenguinlover told me good things about it.

datahead8888 commented 9 years ago

I have successfully built ALL of TSC’s dependencies with MinGW.

@Quintus, what was your reasoning for doing it all through MinGW/MSYS rather than using Windows binaries of the libraries? Is it to avoid the trouble of going to 10,000 some websites and downloading Windows compilations of every open source library or just because that's what MinGW requires?

@Quintus / @dlpenguinlover - What might be the most feasible way to obtain all libraries without building them all? This would be an approach for Visual Studio/Eclipse and I assume any other compiler on Windows that uses dll/lib files or something such as this. chocolatey might be an option for more quickly obtaining some libraries (akin to apt-get); others would have to be downloaded (or possibly still built from source in some cases) from websites.

I have to do most of my schoolwork from Windows, so in reality I probably would find it easier to pop in for a little bit and look at TSC code if I were able to build it from Windows (though I can't promise :) )

Quintus commented 9 years ago

datahead8888 notifications@github.com writes:

I have successfully built ALL of TSC’s dependencies with MinGW.

@Quintus, what was your reasoning for doing it all through MinGW/MSYS rather than using Windows binaries of the libraries? Is it to avoid the trouble of going to 10,000 some websites and downloading Windows compilations of every open source library or just because that's what MinGW requires?

You can’t mix MSVC binaries with MinGW binaries without opening a huge can of problems. See this page for more detailed information:

http://mingw.org/wiki/MixingCompilers

Not all websites provide binaries for MinGW, and while I have to compile some of the dependencies from source, I can also just built the other ones. Personally, I also have experienced (on Linux) problems with linking together binaries produced by different versions of GCC (4.8.x versus 4.9.x in my case), hence I wanted to be sure I do not run into problems that could be avoided.

Vale, Quintus

georgeraraujo commented 9 years ago

I spent a few days on this, and got it to work with MinGW (32-bit mingw-builds i686-4.9.1-release-posix-dwarf-rt_v3-rev2, which came bundled with Qt 5.4.1) on Windows 8.1 x64. I used the Qt Build Suite (Qbs) with Qt Creator, since I don't have a lot of experience with CMake.

Boost 1.57.0 was straightforward to compile.

These libs offer binary downloads, and worked fine out of the box:

These libs don't, and I had to write Qbs project files for them:

These others also don't, and I had to make small changes to their project/build files in order to get them to compile:

While I understand that it would be ideal to get this working with CMake, I got TSC to build and run fine with this setup, and even managed to investigate a bug (#389) or two (change maryo to alex in line 229 of levels/menu_blue_1.tsclvl, line 1948 of levels/menu_brown_1.tsclvl, and line 166 of levels/menu_green_1.tsclvl).

If there's any interest, I could document the steps and upload the project files.

datahead8888 commented 9 years ago

I used the Qt Build Suite (Qbs) with Qt Creator, since I don't have a lot of experience with CMake.

We probably would want to get cmake working for consistency across platforms. I know you can use cmake to generate a Visual Studio solution in Windows. I would assume it can generate a solution that MinGW can use as well. I saw Quintus mentioned a lot of CMakeLists.txt edits will be needed for a native Windows build, though.

I built the dependencies for 0.8.x rather than using the outdated pre-built 0.7.x dependencies

Do you mean you got the game to run with Cegui 0.8? Cegui 0.8 has been driving everyone nuts in #10.

If there's any interest, I could document the steps and upload the project files.

Yes, there's interest. Sharing is good. I probably can't try this myself anytime soon, though, due to time constraints. I know Quintus had looked into this as well (see his above notes). Perhaps some note sharing will finally knock this out.

While I understand that it would be ideal to get this working with CMake, I got TSC to build and run fine with this setup, and even managed to investigate a bug (#389) or two (change maryo to alex in line 229 of levels/menu_blue_1.tsclvl , line 1948 of levels/menu_brown_1.tsclvl , and line 166 of levels/menu_green_1.tsclvl ).

It sounds like we had a bad merge into devel. Merge conflicts have repeatedly happened between the maryo and alex names in .tsclvl files. This would also explain why the issue has only been reported in devel by people. It will be good to get this fixed.

datahead8888 commented 9 years ago

...actually...that bug is strange, if that's happening. If "maryo" does not work in a .tsclvl file, the code in devel may not properly be reverse compatible as we had orginally planned. This means old levels may not load in future version(s) of the game with the current code.

georgeraraujo commented 9 years ago

If I had written CMake project files it would have been just as well - Qt Creator has CMake support built in. It's just that I don't know CMake...

I built CEGUI 0.7.9. What I used from 0.8.x was the dependencies source package, so in the end what I had was CEGUI 0.7.9 built on top of the dependencies for CEGUI 0.8.x. But as far as TSC is concerned, I didn't touch a single line of code.

I'll try and write down the process. It's nothing difficult, but a bit long.

About the bug I reported in the level files, the issue was that Alex never showed up in the title screen, and I got this on stderr.txt: Error loading image : C:/TEMP/share/tsc\pixmaps\maryo/small/stand_right.png Reason : Couldn't open C:/TEMP/share/tsc\pixmaps Error loading GL surface image : C:/TEMP/share/tsc\pixmaps\maryo/small/stand_right.png Reason : Couldn't open C:/TEMP/share/tsc\pixmaps

After I fixed the .tsclvl files, these errors went away, and Alex showed up on the platform to the left of the menu screen.

Quintus commented 9 years ago

I got TSC to build and run fine with this setup, and even managed to investigate a bug (#389) or two (change maryo to alex in line 229 of levels/menu_blue_1.tsclvl , line 1948 of levels/menu_brown_1.tsclvl , and line 166 of levels/menu_green_1.tsclvl ).

Looks to me as if changing him in the menu files just has been forgotten. The menu level files use a static graphic to place him on the platforms. datahead, this is not identical with the actual player figure.

If there's any interest, I could document the steps and upload the project files.

Yes there is. You can either do it here in the tracker, or you might prefer to do it in the wiki. If you’d do it with CMake, we’ll gladly accept it as a Pull Request as well. Once you’re done, I can try to convert your description into CMake instructions also.

so in the end what I had was CEGUI 0.7.9 built on top of the dependencies for CEGUI 0.8.x.

If it works, that’s OK then.

Vale, Quintus

Quintus commented 8 years ago

@georgerbr any chance you get your native windows build documented? @dlpenguinlover state of your efforts?

I ask because it appears there is a probably windows-specific bug at #452, and GDB appears to not work on crosscompiled binaries, so I need a native build.

Valete, Quintus

georgeraraujo commented 8 years ago

I'm working on building with Visual Studio 2013, which has a much bigger installed base on Windows than Qt Creator + MinGW.

I'll let you know when I get this done.

datahead8888 commented 8 years ago

@georgerbr, the task for Visual Studio is #421. It might be better to log your findings there, since this task (#224) has generally been focused on using open source tools for Windows builds. I have abandoned the Visual Studio task for the time being, since I've been focused on other things.

Quintus commented 8 years ago

I managed to build TSC on Windows natively. However, it only works with shared libraries (DLLs), not with all dependencies being linked in statically. Maybe someone more experienced in Windows development than I can figure this out.

As the build system, I used MSYS2. MSYS2 is is a fork of MSYS, and instead of the original MinGW toolchain it uses the MinGW fork mingw-w64, which also supports 64-bit native compilation (but doesn’t really honour Microsoft’s licensing politics as MinGW does). The best thing about MSYS2, though, is it’s terrific package management system based on pacman, the package management system of ArchLinux (!). Their repositories contain a vast number of precompiled software already — including static and dynamic libraries — making dependency management a lot easier. You can basically install all dependencies of TSC except for CEGUI with one command.

CEGUI gieves grief, thogh. It doesn’t detect MSYS2’s FreeImage library, so all I managed to do is to compile it with its own low-performance TGA image codec. I wrote a PKGBUILD for CEGUI 0.7.9 in MSYS2, so you can just feed it into the package builder and it will install all dependencies and build a nice MSYS2 package for you, which you can then install by using pacman. You’ll also need to compile SDL_mixer (I have a PKGBUILD for that as well, but it’s not ready for upload yet). When you’ve done that, simply install all remaining TSC dependencies from MSYS2’s repositories, install cmake, and compile TSC as follows:

$ cmake -G "MSYS Makefiles" -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=$HOME/tsc-test ..
$ make
$ make install

I can post more detailed instructions later, but it does work. However, it currently crashes right after startup with a CEGUI exception. Well, I would have been surprised if any other component caused a problem. It’s always CEGUI if something goes wrong with TSC :-(

Vale, Quintus

Quintus commented 8 years ago

Here’s the SDL_mixer PKGBUILD. I want to submit this to MSYS2 itself when I’ve improved it, but it is enough for us for now.

Valete, Quintus

Quintus commented 8 years ago

Also: The CEGUI PKGBUILD above builds the static version. That’s wrong, you need the shared version. Change --enable-static --disable-shared in the PKGBUILD to the exact opposite.

Valete, Quintus

datahead8888 commented 8 years ago

I am marking this high priority, since it would help us debug #452

georgeraraujo commented 8 years ago

@georgerbr, the task for Visual Studio is #421. It might be better to log your findings there, since this task (#224) has generally been focused on using open source tools for Windows builds. I have abandoned the Visual Studio task for the time being, since I've been focused on other things.

I have put together a dependency package for Windows (uncompressed size is ~700 MB). I tried to use the same versions as MXE. Here's a summary:

I managed to successfully build and run TSC 2.0 with these. I used Qt Creator with MinGW 4.9.2 and Qbs.

I can upload the dependency package (libraries, utilities, and TSC project file) to Dropbox if anyone is interested.

Quintus commented 8 years ago

VS-specific stuff please goes into ticket #421, MinGW-specific stuff here. I change the title accordingly.

Vale, Quintus

Quintus commented 8 years ago

So, here are the detailed instructions for setting up MSYS 2 I promised. If you follow these instructions, you’ll get an environment that will allow you build CEGUI and TSC natively on Windows.

Installing MSYS2

First, you need to install a proper decompression tool like 7Zip that can cope with more advanced compression formats than ZIP.

Next you have to install MSYS2. MSYS2 is a collection of POSIX tools like bash, make, ssh, etc. and most importantly, a Windows port of GCC known as mingw-w64. mingw-w64 is a fork of the original MinGW project that provides newer versions of GCC, the windows APIs, and 64-bit support. It is important to note that unlike Cygwin, the executables created by the MinGW(-w64) compilers are standalone and do not depend on a POSIX layer emulation DLL. This in turn means of course that you can’t use POSIX functionality in your Windows programs behind those offered by Windows itself (which is not much). MSYS2 uses a recent version of Cygwin to provide the environment you work in to make it more appealing to Linux programmers like me, but the compiler produces native Windows executables.

For installing MSYS2, you have to follow the instructions on this page, where you have to choose either the 32-bit or 64-bit variant of MSYS2. Because I only have a 32-bit Windows system, I chose the 32-bit variant. Please download the .tar.xz archive for installation, because if you mess things up, it is easier to reinstall compared to the Windows installer. A quick installation guide (for details please see the linked page):

  1. Extract the archive using 7-Zip so that you end up with a folder C:\msys32.
  2. Open C:\msys32 in the Windows explorer.
  3. Start msys2_shell.bat. As this is the first start, MSYS2 will do some setup work and then ask you to restart the shell. Type exit and then start msys2_shell.bat again.

At this point, you have a bare minimum MSYS2 installation available. MSYS2 offers you two different shells to work with now: The MSYS shell (msys2_shell.bat) and the MinGW shell (mingw32_shell.bat for 32-bit systems or mingw64_shell.bat for 64-bit systems). For managing MSYS (especially for interacting with the package manager) you use the MSYS shell. For compiling software without the package manager involved (like TSC), you use the MinGW shell. More detailed information is available on the MSYS2 wiki. On a side note, the shells MSYS2 provides you don’t run in a CMD window, but in mintty. This means that thinks like resizing the window, copy/paste or ANSI escape sequences will work out of the box, because mintty is a fairly nice terminal emulator.

In the MSYS shell, first update the core system:

$ update-core

If this updates anything, you need to restart the MSYS shell again. Then update the rest of MSYS2:

$ pacman -Su

pacman is MSYS2’s package manager. I am still amazed by this. pacman is software originally developed by ArchLinux as their package management system, and they managed to port it to Windows. So far, I have not found any problems with it; everything I can do on my ArchLinux system with pacman can be done with MSYS2’s pacman on Windows as well. Hence, if you want to know how pacman works, take a look at the ArchWiki and the pacman(8) manpage linked to from that article. (Note: update-core is actually a thin wrapper around pacman that takes care of some Windows specialities)

You now have an up-to-date MSYS2 system.

Installing the software you need

MSYS2 provides a lot of software precompiled via their repositories. Nearly everything you need for TSC can be simply installed with pacman, without having to worry about compilation, which eases the process a lot.

Note that the commands I give below are for a 32-bit system. If you are on a 64-bit Windows system, you need to replace i686 with x86_64 everywhere it occurs.

Execute this in the MSYS shell:

$ pacman -S tar unzip diffutils patch \
  make autoconf automake libtool \
  bison gperf git \
  mingw-w64-i686-cmake \
  mingw-w64-i686-ruby \
  mingw-w64-i686-toolchain \
  mingw-w64-i686-freetype \
  mingw-w64-i686-libxml++ \
  mingw-w64-i686-FreeImage \
  mingw-w64-i686-SDL \
  mingw-w64-i686-SDL_image \
  mingw-w64-i686-SDL_ttf \
  mingw-w64-i686-pcre \
  mingw-w64-i686-boost \
  mingw-w64-i686-glew

The mingw-w64-i686-toolchain package pulls in the GCC compiler, the other names should be pretty self-explanatory.

Installing SDL_mixer

MSYS2 doesn’t have SDL_mixer in the repositories (only SDL2_mixer, which is SDL2). Maybe they just missed it, because I had no problems compiling it; I intend to file a PR against their repository to add it. As TSC depends on SDL_mixer, we need to build it.

To facialate things, I wrote a PKGBUILD for SDL_mixer. PKGBUILDs are build recipes for MSYS2’s package manager; they contain meta information about a package as well as instructions for building them. You can feed them into the makepkg-mingw program, which will then download all required dependencies, build the software and package it for use with pacman. Note that when dealing with PKGBUILDs, you need to use the MSYS shell, and not the MinGW one (think of this as interaction with the package manager).

Download this PKGBUILD and place it in a new directory. In the MSYS shell, navigate into that directory. If you have a 32-bit system like me, you now have to tell MSYS that so it doesn’t attempt to do a 64-bit build which it can’t run:

$ export MINGW_INSTALLS=mingw32

I took this information from this wiki page.

Afterwards, tell pacman to build the software:

$ makepkg-mingw -s

This will download some dependencies and then build SDL_mixer. Note that because I was lazy, this SDL_mixer will only support OGG Vorbis and FLAC music, not MIDI or MP3. As we only use OGG, this shouldn’t be a problem. For the mentioned PR against MSYS2 I’ll have to improve that.

You’ll end up with a package mingw-w64-i686-SDL_mixer.tar.xz. Install it using pacman:

$ pacman -U mingw-w64-i686-SDL_mixer.tar.xz

Installing CEGUI

This is the most problematic step in the whole process. Not because CEGUI is hard to build — it builds just fine —, but getting its build configuration right seems to be impossible. This is the point you’re going to investigate sooner or later if you experiment with native TSC Windows builds.

For quicker success, I wrote a PKGBUILD for CEGUI as well. Repeat the above procedure you did for SDL_mixer now for CEGUI (new directory, executing makepkg-mingw, installing it with pacman). Do not use the same directory you used for SDL_mixer, this will confuse makepkg.

You’ll find some further notes on building CEGUI below, but for now this is enough. CEGUI will build with the OpenGLRenderer, the TGA image codec, and the TinyXML, libxml2, and Expat XML parsers enabled.

Building TSC

exit from the MSYS shell and open the MinGW shell instead now. Create a new directory and clone the TSC repository:

$ git clone git://github.com/Secretchronicles/TSC.git
$ cd TSC
$ git checkout master
$ git submodule init
$ git submodule update

The TSC repository is large, so this will take a while. master is the branch that corresponds to the current stable version, which is 2.0.0 at the moment (you could also check out the v2.0.0 tag, it’s identical). While it is downloading, you could review the INSTALL.md file and see which adaptions it needs. Git might issue some warnings, but I found you can ignore them.

When the download has finished, you need to apply this patch to the sources. It:

Apply it like this (execute in the toplevel TSC directory):

$ patch -p1 < windows.patch
patching file tsc/CMakeLists.txt
patching file tsc/cmake/modules/FindCEGUI.cmake
patching file tsc/mruby_tsc_build_config.rb
patching file tsc/srv/video/video.cpp

Now you can start with the normal TSC build process:

$ cd tsc
$ mkdir build
$ cd build

Execute CMake. Note the -G "MSYS Makefiles" option; without it, CMake would generate MSVC build files, which is inappropriate here. -DCMAKE_BUILD_TYPE=Debug instructs CMake to invoke GCC with the -g option, which in turn causes the for us so important debugging symbols to be included into the final executable. -DCMAKE_INSTALL_PREFIX defines where TSC is going to be installed. The substitution $HOME corresponds to C:\msys32\home\<yourusername> if you followed the instructions above.

$ cmake -G "MSYS Makefiles" -DCMAKE_BUILD_TYPE=Debug \
  -DCMAKE_INSTALL_PREFIX=$HOME/tsc-test ..
-- The CXX compiler identification is GNU 5.2.0
-- Check for working CXX compiler: C:/msys32/mingw32/bin/g++.exe
-- Check for working CXX compiler: C:/msys32/mingw32/bin/g++.exe -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- The C compiler identification is GNU 5.2.0
-- Check for working C compiler: C:/msys32/mingw32/bin/gcc.exe
-- Check for working C compiler: C:/msys32/mingw32/bin/gcc.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Found rake: C:/msys32/mingw32/bin/rake
-- Found gperf: C:/msys32/usr/bin/gperf.exe
-- Found PkgConfig: C:/msys32/mingw32/bin/pkg-config.exe (found version "0.29")
-- Found OpenGL: opengl32
-- Found GLEW: C:/msys32/mingw32/include
-- Found Gettext: C:/msys32/mingw32/bin/msgmerge.exe
-- Found BISON: C:/msys32/usr/bin/bison.exe (found version "3.0.4")
-- Found Freetype: C:/msys32/mingw32/lib/libfreetype.dll.a (found version "2.6.0")
CEGUI.h : C:/msys32/mingw32/include/CEGUI
CEGUIOpenGLRenderer.h : C:/msys32/mingw32/include/CEGUI/RendererModules/OpenGL
Looking for lib: CEGUIOpenGLRenderer
CEGUIOpenGLRenderer : C:/msys32/mingw32/lib/libCEGUIOpenGLRenderer.dll.a
CEGUIBase : C:/msys32/mingw32/lib/libCEGUIBase.dll.a
CEGUIFalagardWRBase : C:/msys32/mingw32/lib/libCEGUIFalagardWRBase.dll.a
CEGUITinyXMLParser : C:/msys32/mingw32/lib/libCEGUITinyXMLParser.dll.a
CEGUITGAImageCodec : C:/msys32/mingw32/lib/libCEGUITGAImageCodec.dll.a
-- Found CEGUI: C:/msys32/mingw32/lib/libCEGUIOpenGLRenderer.dll.a;C:/msys32/mingw32/lib/libCEGUIBase.dll.a;C:/msys32/mingw32/lib/libCEGUIFalagardWRBase.dll.a;C:/msys32/mingw32/lib/libCEGUITinyXMLParser.dll.a;C:/msys32/mingw32/lib/libCEGUITGAImageCodec.dll.a
-- Found LibIntl: C:/msys32/mingw32/lib/libintl.dll.a
-- Boost version: 1.57.0
-- Found the following Boost libraries:
--   filesystem
--   chrono
--   thread
--   system
-- checking for module 'libpng'
--   found libpng, version 1.6.18
-- checking for module 'sdl'
--   found sdl, version 1.2.15
-- checking for module 'SDL_image'
--   found SDL_image, version 1.2.12
-- checking for module 'SDL_mixer'
--   found SDL_mixer, version 1.2.12
-- checking for module 'SDL_ttf'
--   found SDL_ttf, version 2.0.11
-- checking for module 'libpcre'
--   found libpcre, version 8.37
-- checking for module 'libxml++-2.6'
--   found libxml++-2.6, version 2.40.0
-- ----- Path configuration -----
-- Install prefix:   C:/msys32/home/Marvin/tsc-test
-- Data directory:   share/tsc
-- Binary directory: bin
-- (relative pathes are relative to the install prefix)
-- ------------------------------
-- Configuring done
-- Generating done
-- Build files have been written to: C:/msys32/home/Marvin/TSC/tsc/build

Build TSC:

$ make -j4 # Or whatever number of processors you have
$ make install

Cross your fingers and wait. If everything goes well, TSC will compile, and more importantly, link. After make install completes, you will have TSC installed at C:\msys32\home\<yourusername>\tsc-test\bin\tsc.exe.

Running TSC

Still in the MinGW shell, you can execute TSC from the terminal like this:

$ cd ~/tsc-test
$ ./bin/tsc.exe --version
This is The Secret Chronicles of Dr. M. version 2.0.0.
It was compiled from commit c310681.

If that works, all shared libraries are found. You can now start TSC regularyly from the terminal:

$ ./bin/tsc.exe

It will crash with a CEGUI exception that tells you that the TGA image codec fails to load some PNG image:

Initializing resource manager and core classes
Game data directory: C:\msys32\home\Marvin\tsc-test\share\tsc
User data directory: C:/Users/Marvin/AppData/Roaming\tsc
User cache directory: C:/Users/Marvin/AppData/Roaming\tsc\cache
User config directory: C:/Users/Marvin/AppData/Roaming\tsc
Configuration file is 'C:/Users/Marvin/AppData/Roaming\tsc\config.xml'.
Translation locale set to 0 0
Translation locale is German_Germany.1252
Translation support with gettext set to:
        Directory C:\msys32\home\Marvin\tsc-test\share\tsc\translations
        Codeset: UTF-8
        Text domain: Secretchronicles
CEGUI log file is at 'C:/Users/Marvin/AppData/Roaming\tsc\cache\cegui.log'.
CEGUI::RendererException in file CEGUIOpenGLTexture.cpp(139) : OpenGLTexture::loadFromFile - TGAImageCodec - Official TGA image codec failed to load image 'TSCLook512.png'.

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.
CEGUI Scheme Exception occurred : OpenGLTexture::loadFromFile - TGAImageCodec - Official TGA image codec failed to load image 'TSCLook512.png'.
terminate called after throwing an instance of 'CEGUI::RendererException'
  what():  CEGUI::RendererException in file CEGUIOpenGLTexture.cpp(139) : OpenGLTexture::loadFromFile - TGAImageCodec - Official TGA image codec failed to load image 'TSCLook512.png'.

This is the farthest I was able to go.

Notes on CEGUI

CEGUI needs an “image codec” for working properly. These are nothing else than adapter interfaces for some popular image manipulation libraries; CEGUI itself includes such an adapter plus the required library named TGA, but that codec is slow and I suspect buggy. It is thus recommended to build CEGUI with either of the two advanced image codecs, FreeImage or DevIL. You’ll have noticed that the FreeImage library was installed with pacman further above, but I patched its requirement out of TSC in favour of TGA. This is because CEGUI for reasons that are unknown to me is unable to detect FreeImage. I simply cannot understand why it doesn’t. Take this simple C program as an example:

#include <stdio.h>
#include <FreeImage.h>

int main(int argc, char* argv[])
{
  FreeImage_Initialise(FALSE);
  printf("FreeImage version: %s\n", FreeImage_GetVersion());
  FreeImage_DeInitialise();
  return 0;
}

It is possible to compile and run this program without problems like this (in the MinGW shell of course):

$ gcc -Wall foo.c -lfreeimage -o foo.exe
$ ./foo.exe
FreeImage version: 3.16.0

That is, FreeImage is there, and it works. But Autotools fails to detect it! @Luiji, maybe you can take a look at this phenomenon, I am completely clueless. If you need someone with a Windows system to assist you, I found that several people in our IRC channel have one, so you are best asking there. Mathnerd seems to be especially knowledgable here.

If you want to try the alternative library, DevIL, you need to build that one, as it isn’t in MSYS2’s repositories. I wrote a PKGBUILD for that one also, though it wasn’t that easy, because it needed patches to build with a recent GCC and recent libPNG. Also note that the build will fail if libMNG is installed, so you need to remove that one first (or tell DevIL’s build system to ignore it). A tarball containing the necessary PKGBUILD with patches is available here. The two patches in it were taken from the MXE bugtracker and the Debian bugtracker. I didn’t get DevIL to be detected by the CEGUI build scripts either, though I have not tried to link an example program similar to the FreeImage one above.

Actually, there are even more alternatives. CEGUI also includes another image codec, STB, and it supports another image library via a codec, SILLY. I was just too tired to try them. Feel free to do so; for SILLY, you’d need to compile that library yourself also.

Conclusion

I have done what I could, and I got really far. Now it’s just CEGUI making problems. I am sure it is possible to figure this out, but after three days of fiddling with the Windows stuff I am now exhausted. Someone else needs to go the last steps and find out what issue CEGUI has.

If you get any problems with libraries not being found (except for FreeImage and DevIL as mentioned above), please check you are in the correct shell. If you try to compile manually from an MSYS shell, you are building cygwin programs, which is not what you want. Most build scripts will fail to work in that environment. Clear the build directory and switch to the MinGW shell.

/cc @Luiji @dlpenguinlover @sydneyjd

Valete, Quintus

Quintus commented 8 years ago

You are lucky, here’s a PKGBUILD for SILLY, including the necessary patch to get it built with a recent version of libPNG (patch taken from ArchLinux): https://gist.github.com/Quintus/b10a0bee9714cae1a11c

CEGUI’s build system detects SILLY and builds the necessary image codec for it. You’ll need to adapt the CEGUI PKGBUILD to depend on the SILLY package if you want to use that, though. Also don’t forget to adapt TSC’s FindCEGUI.cmake file to look for SILLY instead of TGA.

Vale, Quintus

georgeraraujo commented 8 years ago

VS-specific stuff please goes into ticket #421, MinGW-specific stuff here. I change the title accordingly.

These are all for MinGW.

Quintus commented 8 years ago

I am sorry, I was with my thoughts elsewhere. I will fix my comment in the other ticket.

Quintus commented 8 years ago

FYI, #556.

hgdagon commented 6 years ago

First of all, kudos to @Quintus , I wouldn't figure out to to turn off Boost static libs on my own. Everything below concerns the latest devel branch commit, I haven't tried the master branch yet.

Made very little changes to CMakeLists.txt. After these changes, compilation goes fine, but the resulted executable cannot load some assets, and there's no GUI. The game itself runs, and, if you blindly navigate the menu(Start, right arrow, enter), the first level loads.

Here's the output:

Initializing Package Manager
Warning: Preferences file 'C:/Users/George/AppData/Roaming\tsc\config.xml' does not exist. Using default values.
Setting vertical sync not supported
CEGUI::UnknownObjectException in function 'CEGUI::Image& CEGUI::ImageManager::get(const CEGUI::String&) const' (C:/msys64/home/George/cegui-0.8.7/cegui/src/ImageManager.cpp:261) : Image not defined: hud_itembox_item
No joysticks available
Failed to load image "C:\games\TSC\share\tsc\pixmaps". Reason: Unable to open file
Error loading image : C:\games\TSC\share\tsc\pixmaps\game/alex_l.png

Error loading GL surface image
Error loading image : C:\games\TSC\share\tsc\pixmaps\game/gold_m.png

Error loading GL surface image
Error loading image : C:\games\TSC\share\tsc\pixmaps\game/itembox.png

Error loading GL surface image
Error loading image : C:\games\TSC\share\tsc\pixmaps\menu/items/quit.png

Error loading GL surface image
Failed to open sound file "C:\games\TSC\share\tsc\music\game/menu.ogg" (format not supported)
Failed to open sound file "C:\games\TSC\share\tsc\music\game/menu.ogg" (format not supported)
Error loading image : C:\games\TSC\share\tsc\pixmaps\menu/items/quit.png

Error loading GL surface image
Failed to open sound file "C:\games\TSC\share\tsc\music\game/menu.ogg" (format not supported)
Failed to open sound file "C:\games\TSC\share\tsc\music\game/menu.ogg" (format not supported)
Error loading image : C:\games\TSC\share\tsc\pixmaps\menu/items/quit.png

Error loading GL surface image
Failed to open sound file "C:\games\TSC\share\tsc\music\game/menu.ogg" (format not supported)
Failed to open sound file "C:\games\TSC\share\tsc\music\game/menu.ogg" (format not supported)
Error loading image : C:\games\TSC\share\tsc\pixmaps\menu/items/quit.png

Error loading GL surface image
Failed to open sound file "C:\games\TSC\share\tsc\music\game/menu.ogg" (format not supported)
Failed to open sound file "C:\games\TSC\share\tsc\music\game/menu.ogg" (format not supported)

Now, the reason these files can't be loaded is that they don't actually exist, or they exist in a different location / with a different name, or, as is the case with **menu.ogg, it's a symlink.

But, if that's the only problem, I should text in the menu, right? Yet, there's no text in the menu (options/save/load/start) and no HUD in the game itself.

On a side note, I tried different image codecs (FreeImage, DevIL, SILLY) and different XML Parsers (LibXML, Expat) and the result was almost always the same, except for one of the Image codecs (I think it was SILLY) that CEGUI couldn't detect (something about a module not found).

Another issue is that it munches away on the memory (~600mb). I know it's a 2D game, but doesn't that seem a little too much?

CMakeLists.txt, in case you'd like to try it out.

Quintus commented 6 years ago

Thank you for your thorough investigation. This looks actually promising. I reply to you inline below.

Am 13. Januar 2018 um 06:34 Uhr +0000 schrieb hgdagon:

Everything below concerns the latest devel branch commit, I haven't tried the master branch yet.

Do yourself a favour and don't waste your time on master. The branch is completely outdated as it contains stable 2.0.0. devel represents 2.1.0, which will be the next release. master not even contains the SFML switch or the CEGUI update, and that will drive you to insanity if you try build that. It doesn't even build on Linux anymore since most Linux distros have started to ship libraries compiled with -std=c++11.

The game itself runs, and, if you blindly navigate the menu(Start, right arrow, enter), the first level loads.

Can you actually play the level? I'd be surprised given the errors. A tip: you can use the "-l" commandline option to TSC to skip the main menu and directly load a level. E.g.:

$ ./tsc.exe -l lvl_1

To go directly to lvl_1.tsclvl. The full list of levels is in the repository: https://github.com/Secretchronicles/TSC/tree/devel/tsc/data/levels

Warning: Preferences file 'C:/Users/George/AppData/Roaming\tsc\config.xml' does not exist. Using default values.

This is normal for the first run. It should now have created a file at that path. Note that Windows does not care whether you use forward or backward slashes in file names, it accepts both.

Setting vertical sync not supported

Not relevant, it's a display optimisation.

CEGUI::UnknownObjectException in function 'CEGUI::Image& CEGUI::ImageManager::get(const CEGUI::String&) const' (C:/msys64/home/George/cegui-0.8.7/cegui/src/ImageManager.cpp:261) : Image not defined: hud_itembox_item

I have no idea.

No joysticks available

Normal.

Failed to load image "C:\games\TSC\share\tsc\pixmaps". Reason: Unable to open file Error loading image : C:\games\TSC\share\tsc\pixmaps\game/alex_l.png [...]

Did you run "$ make install" after your build finished? TSC can't currently be run from the build directory, and specifically for Windows, it uses hardcoded pathes relative to the programme executable to find its assets. I.e., you need to run the game executable stored from where "$ make install" placed it. To go extra sure, you might want to change your working directory to the directory containing the installed executable, but that shouldn't be required.

Failed to open sound file "C:\games\TSC\share\tsc\music\game/menu.ogg" (format not supported) Failed to open sound file "C:\games\TSC\share\tsc\music\game/menu.ogg" (format not supported)

The same. You should not hear music as a consequence.

Error loading image : C:\games\TSC\share\tsc\pixmaps\menu/items/quit.png [...]

The menu items in the main menus are images as you probably noticed. If those fail to load, they can't be displayed.

Now, the reason these files can't be loaded is that they don't actually exist, or they exist in a different location / with a different name, or, as is the case with **menu.ogg, it's a symlink.

"make install" should not install symlinks. If it does, it's a bug in the build system. Windows can't do symlinks. As for symlinks in the source repository, shouldn't Git actually take care of replacing a symlink with a copy on Windows when cloning a repository?

The symlink was created when I moved the file to a better organisational place. Since I can't know which levels all over the world reference that music file and can't update all of them (espacially all the custom user levels), I left a symlink where the original was. The TSC code base itself has been updated to the new path.

But, if that's the only problem, I should text in the menu, right? Yet, there's no text in the menu (options/save/load/start) and no HUD in the game itself.

No, it's a natural consequence of TSC being unable to find its assets. The text is drawn using a set of TTF font files of "DejaVu Sans", and if those font files can't be loaded, there will be no text. The font files are here in the repository: https://github.com/Secretchronicles/TSC/tree/devel/tsc/data/gui/fonts

On a side note, I tried different image codecs (FreeImage, DevIL, SILLY) and different XML Parsers (LibXML, Expat) and the result was almost always the same, except for one of the Image codecs (I think it was SILLY) that CEGUI couldn't detect (something about a module not found).

I've provided a PKGBUILD for SILLY further above in the comments some years ago that fixes that problem I think. But CEGUI should work equally well with any image codec it supports.

Another issue is that it munches away on the memory (~600mb). I know it's a 2D game, but doesn't that seem a little too much?

I suspect the memory footprint is so high on Windows, because it links in everything statically. Once you start the executable, the entire thing is loaded up to RAM. Try inspecting the size of the tsc.exe file, it may not be much smaller than that.

The reasons might be different as well; I haven't checked the Linux memory footprint. But I have a piece of advice for you. We as the TSC team have inherited a code base that originates from 2003 when it was stilled called "Secret Maryo Chronicles" (http://www.secretmaryo.org/) where it was started by a hobbyist learning how to programme. Due to this historical heritage the code base contains a nearly infinite number of bad design decisions (you'll be scrared by TSC's massive use of global variables, to just give you a single example; check the "Legacy Code" label on the tracker for more) which have driven us nuts several times. Therefore, don't even think about design decisions that could be improved; the entire design of the code base is just flawed from the roots. A high memory footprint is probably one of the minor problems.

After years of fixing things here and there, just to only discover some new abyss, I have decided that TSC3 will be a rewrite from the ground up. kirbyfan and probably datahead continue their work on TSC2 as they find time, while I focus on the TSC3 rewrite. Currently, however, I'm in the final phase of my studies and thus only work on TSC occasionally. I'm available for any questions, though.

I hope I haven't scrared you. If you manage your way to a working TSC with MSYS2, you can file a PR that I will merge.

-- Blog: https://mg.guelker.eu PGP/GPG ID: F1D8799FBCC8BC4F

hgdagon commented 6 years ago

Sorry for the late answer, I did not check my notifications in a long time. So, let's go one by one

Do yourself a favour and don't waste your time on master. The branch is completely outdated as it contains stable 2.0.0.

Well, It's a little late for that. I wanted to understand the building process a little bit better, so I followed the SMC compiling instructions, It kind of worked, but the resulted executable did not output any sound whatsoever (but the menu, on the other hand showed up just fine). And then I thought I'd compile TSC master, since I now have CEGUI 0.7.9 compiled (which was, actually the hardest thing to do for forever), but, apparently I've been really spoiled with CMake being EVERYWHERE nowadays; I couldn't get the configure script to see my libraries (I only managed to get it to see Boost, but everything else failed.). That's where I stopped, I don't think I'll ever go back to Master.

devel represents 2.1.0, which will be the next release. master not even contains the SFML switch or the CEGUI update, and that will drive you to insanity if you try build that. It doesn't even build on Linux anymore since most Linux distros have started to ship libraries compiled with -std=c++11.

Quite frankly, kudos to you, guys, for adapting SFML, since it made a lot of thing easier, even though at first I did have that stupid problem, where I couldn't get CMake to detect, but, just like I said (#619), it was a lot easier than I thought.

Can you actually play the level? I'd be surprised given the errors.

I have since deleted the build, but I can compile it again if there's other stuff to test, but, yes the levels played just fine, but there was no time counter/coin counter/etc. No text at all, just plain game screen. Also, those errors were being generated while I was switching through the menu (save to load, load to options, etc.), while playing the actual levels, there were no errors.

The menu items in the main menus are images as you probably noticed. If those fail to load, they can't be displayed.

The problem with quit.png, is that it does NOT exist in pixmaps/menu/items, although there is a quit.png in pixmaps/menu, the same is true for all other png errors. Path differences or name differences (i.e. there is a file with a slightly different name in the very same location). So, the game is looking for the files in wrong locations.

"make install" should not install symlinks. If it does, it's a bug in the build system. Windows can't do symlinks. As for symlinks in the source repository, shouldn't Git actually take care of replacing a symlink with a copy on Windows when cloning a repository?

Git did not replace anything, probably because I was using MSYS2's Git package. make install just copied the data folder as it was, with the symlinks. Also, after manually replacing the symlink with the actual ogg file, I DID hear music in the menu.

No, it's a natural consequence of TSC being unable to find its assets. The text is drawn using a set of TTF font files of "DejaVu Sans", and if those font files can't be loaded, there will be no text. The font files are here in the repository: https://github.com/Secretchronicles/TSC/tree/devel/tsc/data/gui/fonts

The data folder, in it's entirety did get copied (I compared with BCompare, all files were in place), I also have The full DejaVu font pack installed in the system. But, if the fonts are the same as they were in SMC, I don't think the system install could've interfered anything, since, the menus displayed just fine in SMC.

I suspect the memory footprint is so high on Windows, because it links in everything statically. Once you start the executable, the entire thing is loaded up to RAM. Try inspecting the size of the tsc.exe file, it may not be much smaller than that.

I don't remember remember the initial size of the executable, but I think was ~15-20mb, also I always stripped it before installing. On top of that, as you can see in CmakeLists.txt, I did disable Boost Static build.

I deleted the build, since I didn't really know what else to test. I guess, I can compile it again to provide some screenshots, if that would help. But that would be it at the moment...

hgdagon commented 5 years ago

Here are the issues on Msys2 as of current commit:

Internal MRuby fails; doesn't like that some path start with C:/ other with /C/. Install Mruby package and use that

As mentioned before(#619) pkg_check_modules doesn't work, use find_package instead, see my changes to CMakeLists.txt

In tsc/src/audio/audio.hpp after #Include <queue> add these 2 lines:

#undef IN
#undef OUT

After these changes the only issue is tsc.exe failes linking (undefined references to pretty much everything), and i don't know what causes it... Will mess around some more. If anything changes, I'll comment back.