Compiling in msys2

[hgdagon]

This is just my experience compiling with msys2 (Spoiler alert: I DID NOT Succeed).

Preliminary configuration:

Go ahead and install everything in the dependency list, except for Python-related stuff (numpy, pillow, pycodestyle, pygments, pylint). Get those with pip, except for pillow. Qt5 is 5GB, so it's gonna take a while.

Get dejavu fonts from here and install. These are unrelated to msys2 (and building in general).

Get the latest Pillow source, 4.3.0 as of now. Extract, cd into it and run (in mingw32 shell): python3 setup.py build_ext --disable-jpeg2000 install

Get pyReadLine with pip, Jasper and dlfcn from repo.

If I didn't forget anything, this should be enough. CMake is also looking for backtrace, gperftools, and inotify. These seem to be optional, so I never bothered with these.

As far as the code goes, my only problem with libopenage was the _MSC_VER checks. You guys should really consider replacing those that look for Windows with something else. I've made some changes (libopenage-msys2.zip), but since I couldn't build openage itself, I can only confirm that libopenage builds with these changes, but I have no idea, if it actually works.

When building openage, I got a bunch of unidentified references to Py* (you can feast your eyes here), which is not surprising, since nothing in openage is being linked against, well, anything python-related. At least I couldn't find one in any of the linklibs.rsp's in openage. Since I know pretty much nothing about Python, I didn't know what to do with this, so this was my dead end... I'm not sure if this is related or not, but CMake, for whatever reason, detected PYTHON_LIBRARIES and PYTHON_LIBRARY as libpython3.6.dll, I manually changed them to libpython3.6.dll.a, that didn't seem to change anything... I noticed that there also seems to be a problem with WinMain/SDL_main in run.exe, -lSDL2Main must be added for Windows.

I guess, this is it. I hope you guys find this useful!

[TheJJ]

Thank you! @tusharpm is working on windows support, so I think he can improve this further.

The preprocessor check matches to mscv, but not to msys2, am I right?

About the Python linking problems, I don't know what a .rsp is. Did we do something wrong on our end?

[tusharpm]

I finally got around to installing msys2 to try this procedure. Here are my preliminary observations:

• I had overestimated the MinGW environment. I assumed it would be closer to a POSIX environment than MSVC. Turns out it isn't all that different.
• There are a few things MinGW can reuse from (what I marked as) MSVC-specific code. Eg. error/stackanalyzer.cpp or util/fslike/directory.cpp.
• Others still need to be modified non-trivially. Eg. util/compiler.cpp [MinGW can provide cxxabi.h, dlfcn.h (but no dladdr), unistd.h, etc].
• The buildsystem is making a few mistakes as @hgdagon pointed out:
  • PYTHON_LIBRARIES is not set to a proper path to either libpython3.6m.dll or libpython3.6[m].dll.a. [Setting either one manually seems to work].
  • PYEXT_LIBRARY (copy of PYTHON_LIBRARIES) is linked to the compiled extensions only if CMake is run for MSVC (buildsystem/python.cmake:116). Changing this line to if(WIN32) works. [This results in a proper linklibs.rsp, and fixes the linker errors reported in the link].
  • The WinMain issue is a weird cython quirk; the program run.cpp uses wmain as entry point (for Windows and python3), which needs to be explicitly mentioned to g++ (specifying -municode in LINK_FLAGS seems to work). I would suggest not using SDL_main - I have no clue what that does.

After all those hacks - mostly manually modifying the build files generated by CMake (tiptoeing around the ones it overwrites 😅 ), I got it to build on MinGW32|msys2. The required patch is in this branch. I'll run the game and packaging scripts later.

[hgdagon]

@tusharpm Thanks for the response! I just saw this and gave it a try; building went smooth; no errors. But, looks like something's wrong... I didn't find any tutorial on how to install it, so I did what I usually do:

mkdir build && cd build
cmake-gui .. //all the CMake shenanigans
make
make install

And here's how it installed it. So, the dll's are in a weird location and run.exe is nowhere to be seen... This is not what was supposed to happen, was it?

[tusharpm]

building went smooth; no errors.

That’s nice. You’ve solved a large piece.

The steps you followed for install are correct, also. There’s probably no tutorial because there’s nothing more to it. The pastebin shows everything expected from the install, I think (yes, we don’t install run.exe - the reason it’s built is mentioned in run.py).

For running, please try: python3 -m openage from your terminal.

[hgdagon]

For running, please try: python3 -m openage from your terminal.

@tusharpm Should I always do that? No actual executable? (Again, I've never done Python)

But anyway, it doesn't work.

[tusharpm]

Should I always do that?

An easier way is to create a shortcut to double-click and get you started. I'm yet to figure out how to do that from the installer package.

No actual executable?

python3 is the executable. It suffices for running openage. Although, the plan is to replace openage.bat (packaged with the Windows installer) with a proper executable eventually.

But anyway, it doesn't work.

I'm sorry to hear that. One workaround you should try is to run the command in <install_dir>/bin directory - from your logs that would be C:/Games/openage/bin/. [Note: this can be set for Windows shortcuts in the "Start In:" field.]

[hgdagon]

run the command in /bin directory

I assumed that, since it's a module, it should detect it anywhere, apparently not...

Anyway, I had problem with OpenGL; OpenAge compained that OpenGL 2.1 is not found (While I'm sure I have OpenGL 3.1). This is a problem I had ever since I got this laptop with Win10, I didn't have this problem on my Win7 laptop (equipped with the same processor, and thus the same videochip). So, I build against the Msys2's OpenGL library, but the resulting application loads opengl32.dll from the system. I'm a bit hesitant to replace my system files, so I simply copy opengl32.dll to the application folder to override, in OpenAge's case C:\msys64\mingw32\lib\python3.6\site-packages\openage\game is the location that seemed to work.

Having said that, OpenAge still didn't load, showing this error. And this is the last thing I do with OpenAge. If I plan to contribute the project, I should start studying Python, otherwise I just feel like a baby playing with stuff I don't understand.

@tusharpm Thanks a lot for all your help and your patience! I'll still keep an eye on the project.

[tusharpm]

It doesn't fail to find the python module (which is actually installed in site-packages). The error observed was because the module failed to find and load libopenage.dll, which is the C++ library. The workaround provides the python module a hint to find it.

Furthermore, I wouldn't recommend modifying any system files. Most things (at least everything I've encountered so far) can be solved without doing that. I think the process looks at PATH to get the fallback opengl32.dll, which ends up being the system one. You can manipulate PATH from your terminal before starting the process.

The error you've linked to, is from GUI (or renderer, perhaps?). @TheJJ might know someone who can help with the specific issue. I'm sorry - I'm not an expert in either.

There's some good news - you've successfully entered C++ land. The python piece is solved. You can contribute to the C++ parts (alongside learning Python - same as me 😃) if you wish.

[VelorumS]

OpenAge still didn't load, showing this error.

Strange, it should give some names in the stack trace. The build is somehow not in the debug mode. Maybe building like cmake -DCMAKE_BUILD_TYPE=Debug will make the stack trace better.

There is also a launch option --gl-debug to trace OpenGL errors. Launching is like that: run game --gl-debug.

[tusharpm]

I take responsibility for the stack trace not being human-readable. That functionality is not implemented for Windows (MSVC or MinGW|msys2).

For debugging further, would adding logs in context_extraction_win32.cpp help?

[simonsan]

For later use this should install the dependencies for mingw-w64 on msys2:

pacman -S cmake make git mingw-w64-x86_64-cmake mingw-w64-x86_64-make mingw-w64-x86_64-clang mingw-w64-x86_64-eigen3 mingw-w64-x86_64-cython mingw-w64-x86_64-libepoxy mingw-w64-x86_64-libogg mingw-w64-x86_64-libpng mingw-w64-x86_64-ttf-dejavu mingw-w64-x86_64-freetype mingw-w64-x86_64-fontconfig mingw-w64-x86_64-harfbuzz mingw-w64-x86_64-SDL2_image mingw-w64-x86_64-SDL2 mingw-w64-x86_64-opusfile mingw-w64-x86_64-opus mingw-w64-x86_64-python mingw-w64-x86_64-python-pygments mingw-w64-x86_64-python-jinja mingw-w64-x86_64-python-pillow mingw-w64-x86_64-python-numpy mingw-w64-x86_64-python-pyreadline mingw-w64-x86_64-python-pip mingw-w64-x86_64-qt5

nyan mingw-w64-x86_64-dlfcn mingw-w64-x86_64-flex

Debugger: mingw-w64-x86_64-gdb

Here is a guide on porting as well: https://www.msys2.org/wiki/Porting/

We should write something in our docs how to build with msys2 and see if we can fix issues coming with building mingw-w64.

[simonsan]

@tusharpm I've got it compiled and started ./run.exe --version inside msys64 where all the dependencies are. Gave me an (maybe) Python 3.8 related error but afaiks it looks okayish.

./run.exe --version
openage  [devmode]
ncurses, opengl
GNU 9.3.0 [ -fdiagnostics-color=auto]
Cython 0.29.16
CI config version NOT SET
Traceback (most recent call last):
  File "run.py", line 15, in init run
    main()
  File "C:\msys64\home\<username>\openage\.bin/openage/__main__.py", line 102, in main
    args = cli.parse_args(argv)
  File "C:/msys64/mingw64/lib/python3.8/argparse.py", line 1768, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "C:/msys64/mingw64/lib/python3.8/argparse.py", line 1800, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "C:/msys64/mingw64/lib/python3.8/argparse.py", line 2006, in _parse_known_args
    start_index = consume_optional(start_index)
  File "C:/msys64/mingw64/lib/python3.8/argparse.py", line 1946, in consume_optional
    take_action(action, args, option_string)
  File "C:/msys64/mingw64/lib/python3.8/argparse.py", line 1874, in take_action
    action(self, namespace, argument_values, option_string)
  File "C:\msys64\home\<username>\openage\.bin/openage/__main__.py", line 30, in __call__
    from .versions.versions import get_version_numbers
ModuleNotFoundError: No module named 'openage.versions.versions'

The build log can be found here: https://pastebin.com/b0inWqw3

Update: Error was due to wrong target ^^

[simonsan]

[ 90%] Linking CXX shared module lzxd-cpython-38.dll
cd /d C:\msys64\home\<username>\openage\openage\build\openage\cabextract && C:\msys64\mingw64\bin\cmake.exe -E cmake_link_script CMakeFiles\openage_cabextract_lzxd.dir\link.txt --verbose=1
C:\msys64\mingw64\bin\cmake.exe -E rm -f CMakeFiles\openage_cabextract_lzxd.dir/objects.a
C:\msys64\mingw64\bin\ar.exe cr CMakeFiles\openage_cabextract_lzxd.dir/objects.a @CMakeFiles\openage_cabextract_lzxd.dir\objects1.rsp
C:\msys64\mingw64\bin\g++.exe   -fdiagnostics-color=auto -O2 -g -DNDEBUG -O3  -shared -o lzxd-cpython-38.dll -Wl,--major-image-version,0,--minor-image-version,0 -Wl,--whole-archive CMakeFiles\openage_cabextract_lzxd.dir/objects.a -Wl,--no-whole-archive @CMakeFiles\openage_cabextract_lzxd.dir\linklibs.rsp
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/9.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: CMakeFiles\openage_cabextract_lzxd.dir/objects.a(lzxd.cpp.obj): in function `openage::log::MessageBuilder::should_format() const':
C:/msys64/home/<username>/openage/openage/libopenage/log/message.h:142: undefined reference to `openage::log::LogSinkList::instance()'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/9.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/home/<username>/openage/openage/libopenage/log/message.h:142: undefined reference to `openage::log::LogSinkList::supports_loglevel(openage::log::level) const'
collect2.exe: error: ld returned 1 exit status
mingw32-make[2]: *** [openage\cabextract\CMakeFiles\openage_cabextract_lzxd.dir\build.make:107: openage/cabextract/lzxd-cpython-38.dll] Error 1
mingw32-make[2]: Leaving directory 'C:/msys64/home/<username>/openage/openage/build'
mingw32-make[1]: *** [CMakeFiles\Makefile2:2614: openage/cabextract/CMakeFiles/openage_cabextract_lzxd.dir/all] Error 2
mingw32-make[1]: *** Waiting for unfinished jobs....
[ 90%] Linking CXX shared module smp-cpython-38.dll
cd /d C:\msys64\home\<username>\openage\openage\build\openage\convert && C:\msys64\mingw64\bin\cmake.exe -E cmake_link_script CMakeFiles\openage_convert_smp.dir\link.txt --verbose=1
C:\msys64\mingw64\bin\cmake.exe -E rm -f CMakeFiles\openage_convert_smp.dir/objects.a
C:\msys64\mingw64\bin\ar.exe cr CMakeFiles\openage_convert_smp.dir/objects.a @CMakeFiles\openage_convert_smp.dir\objects1.rsp
C:\msys64\mingw64\bin\g++.exe   -fdiagnostics-color=auto -O2 -g -DNDEBUG -O3  -shared -o smp-cpython-38.dll -Wl,--major-image-version,0,--minor-image-version,0 -Wl,--whole-archive CMakeFiles\openage_convert_smp.dir/objects.a -Wl,--no-whole-archive @CMakeFiles\openage_convert_smp.dir\linklibs.rsp
mingw32-make[2]: Leaving directory 'C:/msys64/home/<username>/openage/openage/build'
mingw32-make[1]: Leaving directory 'C:/msys64/home/<username>/openage/openage/build'
mingw32-make: *** [Makefile:182: all] Error 2

MSVC on Appveyor says the following to it:

c:\projects\openage\libopenage\error\../log/message.h(58): warning C4251: 'openage::log::message::text': class 'std::basic_string<char,std::char_traits<char>,std::allocator<char>>' needs to have dll-interface to be used by clients of struct 'openage::log::message' (compiling source file C:\projects\openage\libopenage\util\vector_test.cpp) [C:\projects\openage\build\libopenage\libopenage.vcxproj]

Undefined references and linking to DLLs/SOs https://www.msys2.org/wiki/Porting/

Linux/ELF platforms generally don't do anything special to link to shared objects, they just leave the undefined references in the binary. Windows however requires all references to be resolved at link time. In case of DLLs, this is solved by the .dll.a import libraries that add the relevant .dll to the binary's import table and insert correct calls into the code, but it needs that correct linker flags be passed when linking binaries. Note that the linker is aware of these files and will use them automatically when using the standard -l arguments, for example -lfoo will make the linker check for libfoo.dll.a and libfoo.a, in this order (unless specified otherwise).

Libtool generally refuses to create DLLs unless -no-undefined is passed to the linker invocation (library_la_LDFLAGS = -no-undefined). See: https://lists.gnu.org/archive/html/libtool/2007-04/msg00066.html

From the link:

Libtool will not create shared libraries without -no-undefined on PE targets, period. It's not about it trying to detect whether there are or are not any undefined symbols, it's about the maintainer telling libtool that it is safe to assume or not assume something. So yes, you need to either use -no-undefined unconditionally, or conditionalized on PE targets.

C:/msys64/mingw64/lib/libpython3.8.dll.a ../../libopenage/libopenage.dll.a -lkernel32 -luser32 -lgdi32 -lwinspool -lshell32 -lole32 -loleaut32 -luuid -lcomdlg32 -ladvapi32

This is the content of the linklibs.rsp in openage/cabextract.

I already tried -Wno-undef and initialized CC=GCC.

What could it be, how can I diagnose further or am I missing something obvious?

[simonsan]

One step further.

[hgdagon]

I just saw the new page and there's a few issues:

1. These steps are unneeded, there's a dedicated MinGW64 shell

Start a MSYS2-shell from /msys2_shell.cmd

....................................

Add MingW64 Binary Directory To $PATH

Add the /bin-directory for the MingW-W64-Toolchain to $PATH by putting the following command to the shell:

echo 'export PATH=/mingw64/bin:$PATH' >> ~/.bashrc

2. The package list is too long,  some of the packages can be omitted:

`pacman -S --noconfirm --needed flex git mingw-w64-x86_64-toolchain mingw-w64-x86_64-cmake mingw-w64-x86_64-eigen3 mingw-w64-x86_64-cython mingw-w64-x86_64-dlfcn mingw-w64-x86_64-libepoxy mingw-w64-x86_64-fontconfig mingw-w64-x86_64-SDL2_image mingw-w64-x86_64-opusfile mingw-w64-x86_64-python-pycodestyle mingw-w64-x86_64-python-pip mingw-w64-x86_64-python-pillow mingw-w64-x86_64-python-pylint mingw-w64-x86_64-python-numpy mingw-w64-x86_64-cython mingw-w64-x86_64-python-pygments mingw-w64-x86_64-python-jinja mingw-w64-x86_64-python-pyreadline mingw-w64-x86_64-qt5`

I added `pycodestyle` and `pylint`, since those are in the list, but if those are unneeded, they can be removed to the list, they don't have any unique dependencies. `mingw-w64-x86_64-toolchain` installs the whole MinGW64 toolchain with all it's dependencies. I added `flex` and `dlfcn` to download `nyan` automatically, since it's not in the repo.

3. A little cheat here:

cmake -DCMAKE_BUILD_TYPE=Debug -DCMAKE_MAKE_PROGRAM=mingw32-make -Wno-dev -G "CodeBlocks - MinGW Makefiles" .. mingw32-make -j$(nproc)

Ususlly with cmake, I cheat and create a duplicate make.exe:

cp /mingw64/bin/mingw32-make.exe /mingw64/bin/make.exe

This eliminates the need to change make program from command line.

I'm also not sure why you're using "CodeBlocks - MinGW Makefiles", "MSYS Makefiles" is what we need. MinGW assumes we're using cmd.exe, msys will use the shell.

However, with all of these changes I still couldn't test OpenAge, since nyan building failed.
I included `string` in `nyan-external\source\nyan\nyan_tool.h` and `stdint.h` in `libopenage\util\hash.h`, but even after all these changes, linking lzxd-cpython-38.dll still fails.I'm sure I'm missing another dependency cabextract, I just don't know what it is.

[simonsan]

1. These steps are unneeded, there's a dedicated [MinGW64 shell](https://www.msys2.org/wiki/MSYS2-introduction/#subsystems)

Jop, might be better to use this for the usecase of building. (now I know what you meant, yeah, will change that)

1. The package list is too long,  some of the packages can be omitted:

I personally prefer to not install things that are unneeded. The toolchain would install that. If you are talking about the PR it is still work in progress so the best development workflow is still to be figured out.

1. A little cheat here:

Cheating is cheating. ;-) Let's keep it clean.

I'm also not sure why you're using "CodeBlocks - MinGW Makefiles", "MSYS Makefiles" is what we need. MinGW assumes we're using cmd.exe, msys will use the shell.

I don't think so. We want to use cmd.exe (due to the mingw32-make approach) https://cmake.org/cmake/help/v3.17/generator/MinGW%20Makefiles.html#generator:MinGW%20Makefiles

However, with all of these changes I still couldn't test OpenAge, since nyan building failed.

With my current toolchain I can build libnyan, libopenage fine. Also linking with all the libraries for python works. The only problem I currently see is Python 3.8 add_dll_directory function. So starting openage is still not working.

[hgdagon]

I personally prefer to not install things that are unneeded. The toolchain would install that

You can replace toolchian with gcc, it doesn't have any extra dependencies that we need.

We want to use cmd.exe (due to the mingw32-make approach)

I really don't understand. You can use mingw32-make or make with the msys shell. You can either pass -DCMAKE_MAKE_PROGRAM=mingw32-make, which is your current approach, you can use my cheat, or, since you'd prefer to go clean, you can just install make. pacman -S make All of these can work in the shell. I just don't understand why start with msys2, if we're gonna ditch it midway, and use cmd.exe.

If you think that you have to use cmd.exe, bc it ships with mingw32-make, that's wrong. That page is referring to the MinGW project, we're using msys2, and MinGW (if installed from msys2 repo) works very well in the shell.

With my current toolchain I can build libnyan

I forgot to strike that line through. As I mentioned, I had to add a couple if includes for everything to build, but linking lzxd-cpython-38.dll fails.

A few word on warnings, in case anybody would like eliminate these. I received the following warnings during build:

libopenage/gui/guisys/private/livereload/gui_live_reloader.cpp (qmlRegisterType is deprecated: Use qmlRegisterAnonymousType instead)
libopenage/util/compiler.cpp (unused parameter 'require_exact_addr')
libopenage/util/os.cpp (unused parameter 'path')
libopenage/util/os.cpp (unused parameter 'background')
libopenage/util/subprocess.cpp (unused parameter 'filename')
libopenage/util/subprocess.cpp (unused parameter 'argv')
libopenage/util/subprocess.cpp (unused parameter 'wait')
libopenage/util/subprocess.cpp (unused parameter 'redirect_stdout_to')
libopenage/util/fslike/directory.cpp ("NOMINMAX" redefined)
openage/convert/slp.pyx (Using depricated NumPy API, disable it with #define NPY_NO_DEPRICATED_API NPY_1_7_API_VERSION)
openage/convert/smx.pyx (Using depricated NumPy API, disable it with #define NPY_NO_DEPRICATED_API NPY_1_7_API_VERSION)
openage/convert/smp.pyx (Using depricated NumPy API, disable it with #define NPY_NO_DEPRICATED_API NPY_1_7_API_VERSION)

[TheJJ]

Thx for telling me about the include issues, I've added them in #1268. Can you please post the error why linking lzxd fails?

[simonsan]

@hgdagon Feel free to add a review in the PR so we have everything in one place, I pretty much just started this because no one else did. I'm working on something else right now, so I would also appreciate it, if you add your information there for example. ;-)

[hgdagon]

@TheJJ here's my linking error:

C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.1.0/../../../../x86_64-w64-mingw32/bin/ld.exe: CMakeFiles/openage_cabextract_lzxd.dir/objects.a(lzxd.cpp.obj): in function `openage::log::MessageBuilder::should_format() const':
C:/msys64/home/George/openage/libopenage/log/message.h:142: undefined reference to `openage::log::LogSinkList::instance()'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.1.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/home/George/openage/libopenage/log/message.h:142: undefined reference to `openage::log::LogSinkList::supports_loglevel(openage::log::level) const'
collect2.exe: error: ld returned 1 exit status
make[2]: *** [openage/cabextract/CMakeFiles/openage_cabextract_lzxd.dir/build.make:106: openage/cabextract/lzxd-cpython-38.dll] Error 1
make[1]: *** [CMakeFiles/Makefile2:2710: openage/cabextract/CMakeFiles/openage_cabextract_lzxd.dir/all] Error 2
make: *** [Makefile:183: all] Error 2

@simonsan Which PR? You have several.

[simonsan]

@hgdagon The one called Building on MSYS2 with MingW-W64 (Link: #1236).

[simonsan]

C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.1.0/../../../../x86_64-w64-mingw32/bin/ld.exe: CMakeFiles/openage_cabextract_lzxd.dir/objects.a(lzxd.cpp.obj): in function openage::log::MessageBuilder::should_format() const': C:/msys64/home/George/openage/libopenage/log/message.h:142: undefined reference toopenage::log::LogSinkList::instance()' C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.1.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/home/George/openage/libopenage/log/message.h:142: undefined reference to `openage::log::LogSinkList::supports_loglevel(openage::log::level) const' collect2.exe: error: ld returned 1 exit status make[2]: [openage/cabextract/CMakeFiles/openage_cabextract_lzxd.dir/build.make:106: openage/cabextract/lzxd-cpython-38.dll] Error 1 make[1]: [CMakeFiles/Makefile2:2710: openage/cabextract/CMakeFiles/openage_cabextract_lzxd.dir/all] Error 2 make: *** [Makefile:183: all] Error 2 ``

This error should be fixed, it's because the class/function is not exported/exposed to the OAAPI. Should be fixed here.

[hgdagon]

@hgdagon The one called Building on MSYS2 with MingW-W64 (Link: #1236).

@simonsan I added a review. First time reviewing, so the quotes aren't really correct, I apologize.

This error should be fixed, it's because the class/function is not exported/exposed to the OAAPI. Should be fixed here.

I build that branch, and here's how it installed:

Since I didn't change the install prefix in Cmake, I redirected the install path C:\Games\OpenAge to see what's being installed.

-- Installing: C:/Games/OpenAge/Program Files (x86)/openage/share/pixmaps/openag
e.svg ------------Do we need this on Windows?
-- Installing: C:/Games/OpenAge/Program Files (x86)/openage/share/openage/banner
.svg ------------Or this?
-- Installing: C:/Games/OpenAge/Program Files (x86)/openage/share/openage/shader
s/.
-- Installing: C:/Games/OpenAge/Program Files (x86)/openage/share/openage/shader
s/./alphamask.frag.glsl
-- Installing: C:/Games/OpenAge/Program Files (x86)/openage/share/openage/shader
s/./alphamask.vert.glsl
-- Installing: C:/Games/OpenAge/Program Files (x86)/openage/share/openage/shader
s/./equalsEpsilon.glsl
-- Installing: C:/Games/OpenAge/Program Files (x86)/openage/share/openage/shader
s/./identity.vert.glsl
-- Installing: C:/Games/OpenAge/Program Files (x86)/openage/share/openage/shader
s/./maptexture.frag.glsl
-- Installing: C:/Games/OpenAge/Program Files (x86)/openage/share/openage/shader
s/./maptexture.vert.glsl
-- Installing: C:/Games/OpenAge/Program Files (x86)/openage/share/openage/shader
s/./teamcolors.frag.glsl
-- Installing: C:/Games/OpenAge/Program Files (x86)/openage/share/openage/shader
s/./texturefont.frag.glsl
-- Installing: C:/Games/OpenAge/Program Files (x86)/openage/share/openage/shader
s/./texturefont.vert.glsl
-- Installing: C:/Games/OpenAge/Program Files (x86)/openage/share/openage/textur
es/.
-- Installing: C:/Games/OpenAge/Program Files (x86)/openage/share/openage/textur
es/./torn_paper_edge.docx
-- Installing: C:/Games/OpenAge/Program Files (x86)/openage/share/openage/textur
es/./torn_paper_edge.png  ------------Note the dots.
-- Installing: C:/Games/OpenAge/Program Files (x86)/openage/share/openage/qml
-- Installing: C:/Games/OpenAge/Program Files (x86)/openage/share/applications/o
penage.desktop  ------------Definitely don't need this.
-- Installing: C:/Games/OpenAge/etc/openage/keybinds.oac ------------So, this goes to C:\etc, need to redirect.
-- Installing: C:/Games/OpenAge/Program Files (x86)/openage/lib/cmake/nyan/
....
-- Installing: C:/Games/OpenAge/Program Files (x86)/openage/lib/libnyan.dll.a
-- Installing: C:/Games/OpenAge/Program Files (x86)/openage/lib/libnyan.dll
-- Installing: C:/Games/OpenAge/Program Files (x86)/openage/include/nyan -------lib and include should also not be installed.
-- Installing: C:/Games/OpenAge/Program Files (x86)/openage/bin/nyancat.exe -----not sure about this
-- Installing: C:/Games/OpenAge/Program Files (x86)/openage/bin/libopenage.dll
-- Installing: C:/Games/OpenAge/Program Files (x86)/openage/bin/libnyan.dll
-- Installing: C:/Games/OpenAge/msys64/mingw64/lib/python3.8/site-packages/opena
ge/....... 

However, running this is still a mystery, since, even if everything is in place, it won't run.

[simonsan]

However, running this is still a mystery, since, even if everything is in place, it won't run.

My assumption is that it won't run because of #1201

[hgdagon]

@simonsan Actually, this is the error I get:

Traceback (most recent call last):
  File "C:/msys64/mingw64/lib/python3.8/runpy.py", line 194, in _run_module_as_m
ain
    return _run_code(code, main_globals, None,
  File "C:/msys64/mingw64/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\msys64\home\George\openage\build/openage/__main__.py", line 140, in <
module>
    sys.exit(main())
  File "C:\msys64\home\George\openage\build/openage/__main__.py", line 133, in m
ain
    return args.entrypoint(args, cli.error)
  File "C:\msys64\home\George\openage\build/openage/game/main.py", line 32, in m
ain
    from .main_cpp import run_game
ModuleNotFoundError: No module named 'openage.game.main_cpp'

Which is correct, I don't have main_cpp, I have main_cpp-cpython-38.pyd. And, looks like, this is the cause.

I don't have enough time to keep an eye on this all the time, but thank you. I started this issue 3 years ago, and it's nice to see someone working on this.

[simonsan]

Which is correct, I don't have main_cpp, I have main_cpp-cpython-38.pyd. And, looks like, this is the cause.

It was a proposal of @TheJJ and I think as well that it's part of a fix. https://docs.python.org/3/faq/windows.html?highlight=pyd#is-a-pyd-file-the-same-as-a-dll

But did you try if you rename it to main_cpp.pyd if it works then?

[hgdagon]

It was a proposal of @TheJJ and I think as well that it's part of a fix.

Still, it generates wrong filenames.

But did you try if you rename it to main_cpp.pyd if it works then?

Yes. I had to rename quite few files, actually. It ran, asked for AoE2 directory, went through conversion, but running failed:

INFO [py] launching openage
INFO [py] compiled by GNU 10.1.0
INFO launching engine with Path(Union(<openage.util.fslike.union.Union object at 0x000000001b921ca0>.root @ (b'cfg',), <openage.util.fslike.union.Union object at 0x000000001b921910>.root @ (b'assets',)):) and fps limit 0
INFO SDL audio subsystems initialized
INFO Using audio device: default [freq=48000, format=32784, channels=2, samples=4096]
INFO Initialized SDL video subsystems.
Traceback (most recent call last):
  File "openage", line 15, in <module>
    main()
  File "C:\msys64\mingw64\lib\python3.8\site-packages\openage\__main__.py", line 133, in main
    return args.entrypoint(args, cli.error)
  File "C:\msys64\mingw64\lib\python3.8\site-packages\openage\game\main.py", line 78, in main
    return run_game(args, root)
  File "openage/game/main_cpp.pyx", line 47, in openage.game.main_cpp.run_game
  File "", line 0, in PyInit_main_cpp
  File "", line 0, in ZN7openage8run_gameERKNS_14main_argumentsE
  File "", line 0, in
  File "", line 0, in ZN7openage5error5ErrorC2ERKNS_3log7messageEbb
  File "libopenage/engine.cpp", line 161, in openage::Engine::Engine(const openage::util::Path&, int32_t, bool, const char*)
openage.cppinterface.exctranslate.CPPException: could not find qml root folder Path(Union(<openage.util.fslike.union.Union object at 0x000000001b921ca0>.root @ (b'cfg',), <openage.util.fslike.union.Union object at 0x000000001b921910>.root @ (b'assets',)):/assets/qml)

@simonsan

[simonsan]

It seems like the installation procedure is not working out for the way the filesystem is setup on msys. Did you start it from cmd or from msys shell?

Furthermore I wonder if you maybe started it from outside but had a different Python version in the PATH. Assuming that from the [36m[py] remark in the log, whereas you compiled with Python 3.8 on msys

[heinezen]

Furthermore I wonder if you maybe started it from outside but had a different Python version in the PATH. Assuming that from the [36m[py] remark in the log, whereas you compiled with Python 3.8 on msys

@simonsan Those are ANSI sequences for coloring the terminal output :)

https://en.wikipedia.org/wiki/ANSI_escape_code#Colors

[hgdagon]

It seems like the installation procedure is not working out for the way the filesystem is setup on msys. Did you start it from cmd or from msys shell? Furthermore I wonder if you maybe started it from outside but had a different Python version in the PATH. Assuming that from the [36m[py] remark in the log, whereas you compiled with Python 3.8 on msys

I started from cmd, but Python for Windows is not in my path, so it's still using msys2's python. I tried running in the shell via python -m openage, now I'm seeing that ImportError, but I can't do anything about it. I don't know where add_dll_directory() should go (main.py?), and, even if I did, I don't know the syntax.

So, yeah, I shouldn't have run it from cmd, in the first place.

[TheJJ]

In #1301 I updated the dll -> pyd renaming, so maybe your import error is now gone. Could you check, please? It should crash shortly afterwards thoug, due to a different error (probably missing files etc).

[hgdagon]

@TheJJ I don't know what this is. Googling says it doesn't exist on Windows.

D:/msys64/home/George/openage/build/openage/convert/service/export/png/png_create.cpp: In function 'PyObject* __pyx_f_7openage_7convert_7service_6export_3png_10png_create_optimize_greedy(__Pyx_memviewslice, int, int, __pyx_t_7openage_7convert_7service_6export_3png_10png_create_greedy_replay_param)':
D:/msys64/home/George/openage/build/openage/convert/service/export/png/png_create.cpp:3491:16: error: 'open_memstream' was not declared in this scope
 3491 |   __pyx_v_fp = open_memstream((&__pyx_v_outbuffer), (&__pyx_v_outbuffer_len));
      |                ^~~~~~~~~~~~~~
D:/msys64/home/George/openage/build/openage/convert/service/export/png/png_create.cpp: In function '__pyx_t_7openage_7convert_7service_6export_3png_10png_create_greedy_replay_param __pyx_f_7openage_7convert_7service_6export_3png_10png_create_optimize_greedy_iterate(__Pyx_memviewslice, int, int)':
D:/msys64/home/George/openage/build/openage/convert/service/export/png/png_create.cpp:3842:24: error: 'open_memstream' was not declared in this scope
 3842 |           __pyx_v_fp = open_memstream((&__pyx_v_buf), (&__pyx_v_len));
      |                        ^~~~~~~~~~~~~~
make[2]: *** [openage/convert/service/export/png/CMakeFiles/openage_convert_service_export_png_png_create.dir/build.make:83: openage/convert/service/export/png/CMakeFiles/openage_convert_service_export_png_png_create.dir/png_create.cpp.obj] Error 1
make[1]: *** [CMakeFiles/Makefile2:3509: openage/convert/service/export/png/CMakeFiles/openage_convert_service_export_png_png_create.dir/all] Error 2

This happens @ 90% during the build process.

[heinezen]

@hgdagon Oh no, I knew this would haunt us sooner or later :/

Edit: Opened issue #1306 addressing this.

[lbccjyx]

File "openage/game/main_cpp.pyx", line 47, in openage.game.main_cpp.run_game this file named main_cpp.pyx may be wrong. change it to main_cpp.pyd again?

I also meet this problem. ImportError: DLL load failed while importing main_cpp I rename "main_cpp.cp310-win_amd64.pyd" to "main_cpp.pyd"
rename "main_cpp.cp310-win_amd64.pyd.recipe" to "main_cpp.pyd.recipe" rename "main_cpp.cp310-win_amd64.ilk" to "main_cpp.ilk" but not useful