search

clementgallet / libTAS

GNU/Linux software to (hopefully) give TAS tools to games
GNU General Public License v3.0
493 stars 58 forks source link

Wine Compatibility #54

Closed ACrowIAm closed 5 years ago

ACrowIAm commented 6 years ago

Please add a feature to use Wine so we can tas .exe games like I Wanna Be The Boshy and games like that because i tested Wine with IWBTB and it ran perfectly no crashes. I think this is a very cool idea because hourglass for windows isn't very good i had to go on windows xp with no internet to get it to run the game but still it was very buggy.

clementgallet commented 6 years ago

I've tested a bit launching a game with wine, and as expected it is not working.

It's difficult to dig into it because multiple processes are launched when executing wine, and also because wine is using its own library loading code, by-passing linux library loading. However, some people manage to hook OpenGL functions using LD_PRELOAD with wine.

If launch wine with libTAS, I'm getting the following error: wine: cannot find '/home/clement/Desktop/SuperMeatBoy/SuperMeatBoy.exe'. And yes, this file exists. By enabling WINEDEBUG option and checking libTAS logging:

  0.000:0107:0108:trace:file:RtlSetCurrentDirectory_U curdir now L"unix\\home\\clement\\libtas-1.3.0\\build\\" 0x8
  0.000:0107:0108:trace:process:init_current_directory starting in L"unix\\home\\clement\\libtas-1.3.0\\build\\" 0x8
  0.000:0107:0108:trace:nls:MultiByteToWideChar cp 65010 "/home/clement/Desktop/SuperMeatBoy/SuperMeatBoy.exe\x00" -> (null), ret = 52
  0.000:0107:0108:trace:nls:MultiByteToWideChar cp 65010 "/home/clement/Desktop/SuperMeatBoy/SuperMeatBoy.exe\x00" -> L"/home/clement/Desktop/SuperMeatBoy/SuperMeatBoy.exe\0000", ret = 52
  0.000:0107:0108:trace:nls:WideCharToMultiByte cp 0 L"/home/clement/Desktop/SuperMeatBoy/SuperMeatBoy.exe\0000" -> (null), ret = 52
  0.000:0107:0108:trace:nls:WideCharToMultiByte cp 0 L"/home/clement/Desktop/SuperMeatBoy/SuperMeatBoy.exe\0000" -> "/home/clement/Desktop/SuperMeatBoy/SuperMeatBoy.exe\x00", ret = 52
  0.000:0107:0108:trace:file:RtlDosPathNameToNtPathName_U_WithStatus (L"/home/clement/Desktop/SuperMeatBoy/SuperMeatBoy.exe",0x7fffffffc510,(nil),(nil))
  0.000:0107:0108:trace:file:RtlGetFullPathName_U (L"/home/clement/Desktop/SuperMeatBoy/SuperMeatBoy.exe" 520 0x7fffffffc1b0 (nil))
[libTAS f:0] Thread 24016 (main) __lxstat call with path /home/clement/.wine/dosdevices/unix
[libTAS f:0] Thread 24016 (main) __xstat call with path /home/home/clement/Desktop/SuperMeatBoy/SuperMeatBoy.exe
  0.000:0107:0108:trace:file:wine_nt_to_unix_file_name L"\\home\\home\\clement\\Desktop\\SuperMeatBoy\\SuperMeatBoy.exe" not found in /home/home/clement/Desktop/SuperMeatBoy/SuperMeatBoy.exe
  0.000:0107:0108:warn:ntdll:NtQueryAttributesFile L"\\??\\unix\\home\\home\\clement\\Desktop\\SuperMeatBoy\\SuperMeatBoy.exe" not found (c0000034)
  0.000:0107:0108:trace:file:RtlGetFullPathName_U (L"/home/clement/Desktop/SuperMeatBoy/SuperMeatBoy.exe" 520 0x7fffffffcbe0 0x7fffffffc820)
wine: cannot find '/home/clement/Desktop/SuperMeatBoy/SuperMeatBoy.exe'

Somehow the real check (using __xstat which is the function behind stat) checks for /home/home/...

vadosnaprimer commented 6 years ago

I asked ais523 for a wild guess, and this was the reply:

ais523: feos: there's a difference between a directory and a directory name, so one guess is that something's using names to identify directories when it should be using something more permanent (that's, e.g., stable against renames); however, I suspect it's more likely to just be a simply logic error somewhere ais523: such as someone writing "/home/$HOME" rather than "$HOME" or "/home/$LOGNAME"

Don't judge if it doesn't make sense or look relevant.

clementgallet commented 6 years ago

I honestly don't know. But it may hides bigger problems. I'll try to build a custom version of wine by modifying the wine preload code to see if it can load our library first.

ACrowIAm commented 6 years ago

Awesome sounds good,

ACrowIAm commented 6 years ago

Is there any updates on whats going on?

clementgallet commented 5 years ago

Sorry, this is going to be a serious issue, and there is still much to do on current supported games, so I don't feel I'll work on this for now.

InfamousKnight commented 5 years ago

So I’m curios, do you have any clue how this would be implemented?

clementgallet commented 5 years ago

No, we would first need to understand what wine is doing exactly, but it is such a big project.

clementgallet commented 5 years ago

Looking at this again. I managed to hook a wine game correctly, but I'm having a crash at the first created thread. wine_smb

the original pthread_create() is crashing, and it has to do with wine using a specific stack by calling pthread_attr_setstack(). If I'm removing this call from wine, the thread is created correctly, and I'm getting to the first OpenGL screen draw (then I get an Xlib error). However, I think that removing this call may break wine execution at some point. I tried increasing the stack size, and change the memory protection flags, but no luck.

madewokherd commented 5 years ago

I'm a Wine developer. Anything I can do to help debug this?

clementgallet commented 5 years ago

Hey. Thanks for your help!

I've pushed the wine branch with all the specific development on supporting wine. Noop'ing the pthread_attr_setstack() does not seem to bother wine (with probably increased memory usage).

There's a bunch of new stuff to implement, because winex11.drv is using a large portion of the Xlib API and extensions.

I had to disable (3159e2127dcb6118bbec3ca5d0226ac74961b9ee) the custom way for wine to handle window focus (it disables input focus on the window, and intercept the WM_TAKE_FOCUS ClientMessage event to manually give focus to the window). I'm not sure what is the reason for this, but it completely defeats libTAS inputs handling.

One issue I didn't look at it much is that libTAS's HUD does not work. I'm getting those errors:

002c:err:d3d:wined3d_debug_callback 0x7e34ec0: "GL_INVALID_OPERATION in glBindTexture(target mismatch)".
002c:err:d3d:wined3d_debug_callback 0x7e34ec0: "GL_INVALID_OPERATION in glTexImage2D(immutable texture)".
002c:err:d3d:wined3d_debug_callback 0x7e34ec0: "GL_INVALID_OPERATION in glFramebufferTexture2D(mismatched texture target)".

I'm creating a texture and a fbo just before the first draw (glXSwapBuffers), then at each draw, I attach the texture to the fbo and blit it on screen. I suspect there's some context switching done by wine, making my identifiers invalid. I'm really bad at OpenGL coding, so I'm not wiling to look at it now.

The games I've tested so far that work are Super Meat Boy, Dustforce, FTL and The Messenger. Environmental Station Alpha does not work, probably because the executable extracts the Fusion runtime inside a temp directory and execute the runtime on a new process.

madewokherd commented 5 years ago

As I understand it, the alternate stack is there to support the specific memory layout used by Windows, as well as the thread layout information in the Thread Environment Block. I expect this to break exception handling, and I'm not sure what else might be affected.

I think it's OK to disable WM_TAKE_FOCUS. In fact, we have a registry option for it: https://github.com/wine-mirror/wine/blob/master/dlls/winex11.drv/x11drv_main.c#L371 On Win32 it is possible to reject window activation by returning MA_NOACTIVATE from the WM_ACTIVATE message, and WM_TAKE_FOCUS makes it possible to support this feature. Wine tracks window focus internally, so this shouldn't break anything, it'll just mean X11's idea of window focus could get out of sync with Wine.

My OpenGL skills are nonexistent, but I think we do context switching so that an application can use Direct3D and OpenGL API's at the same time without wined3d affecting the GL context.

What is the purpose of the X11 hooking? If it has to do with saving/restoring state, this may not be a workable path. Window HWND handles and related information are duplicated outside the process in wineserver, as are kernel objects like file handles. I think some work will be needed inside Wine to preserve these.

clementgallet commented 5 years ago

X11 hooking is used for:

The data regarding the connection with the X server is one of the few things that must not be restored when loading a savestate, because the sequence numbers must match between the client and the server. So it's ok for wineserver to hold informations about the connection to the server. Actually, savestates are working right now, I had a couple of crashes, but I didn't test it much.

The support of file handles in libTAS is: we don't close files that were opened during a savestate, and we store the file position. After loading the state, the file handle is valid and we just have to set the position.

madewokherd commented 5 years ago

The wineserver doesn't have any X11 data, it's all win32 data. I guess you'd want to do something similar with any win32 handles: don't close them if they were open during a savestate, and retain information like file position.

madewokherd commented 5 years ago

I'm trying Super Hexagon (Linux/64-bit) on the wine branch just to have a good starting point before trying wine. It crashes on startup. Is that expected to work on the branch?

clementgallet commented 5 years ago

No non-wine games are currently working on wine branch, because I set up a env variable to skip the first libTAS library initialization. You can remove it manually by commenting in src/library/main.cpp all the section concerning LIBTAS_DELAY_INIT (line 55 to 62).

qixils commented 5 years ago

do any libTAS settings need to be changed? quickly tested with cat planet (free game if you need more material to test) but libTAS just closed when trying to hit start. (i'm on the wine branch too and using /usr/bin/wine64 as i386 was missing deps)

madewokherd commented 5 years ago

I haven't had any more success than you so far, but I think you'll need to choose wine or wine64 based on the architecture of the exe, so that libTAS can guess the architecture correctly. It seems the configure script has an --enable-i386 switch to build the 32-bit library.

qixils commented 5 years ago

ah! yes, you seem to be right! was able to get the game to launch but didn't get much farther than that, froze while loading in different spots, but i guess that's to be expected at this early stage of work :p

clementgallet commented 5 years ago

Should be easier to use with 941a922c5cd2137c61d983621ef9b0f080be2deb. You can now specify directly Windows binaries in the game executable field. It will detect based on the file output that it's a PE32[+] format and call wine[64] on it, with the appropriate settings.

Managed to get cat planet to launch. Savestates crash often, otherwise I didn't get any issue.

I forgot that there is one needed setting in wine. You need to open winetricks, then select "Select the default wineprefix", "Change settings" and check "sound=alsa".

clementgallet commented 5 years ago

HUD was fixed by initializing HUD texture/fbo on the first draw instead of on context creation (b73f61576d7002acca6e008a9a584f5505428174)

InfamousKnight commented 5 years ago

Yay, Ill be testing out some steam games tomorrow. And some non steam games too.

2 days off of work tomorrow!

InfamousKnight commented 5 years ago

Alright, I tried installing 32 bit depencies by adding :i386 at the end but with libswresample2 I cant seem to get a 32bit version. I believe I need it for midnight club 2 pc demo

InfamousKnight commented 5 years ago

Bump after running this command: sudo apt-get install pkg-config:i386 libx11-dev:i386 libsdl2-dev:i386 libudev-dev:i386 libasound2-dev:i386 libswresample-dev:i386 ffmpeg:i386 this shows up eventually:

dpkg: dependency problems prevent configuration of libmirclient-dev:i386: libmirclient-dev:amd64 (1.1.1-0ubuntu3) breaks libmirclientcpp-dev and is installed. libmirclient-dev:i386 (1.1.1-0ubuntu3) provides libmirclientcpp-dev.

dpkg: error processing package libmirclient-dev:i386 (--configure): dependency problems - leaving unconfigured dpkg: dependency problems prevent configuration of libsdl2-dev:i386: libsdl2-dev:i386 depends on libmirclient-dev; however: Package libmirclient-dev:i386 is not configured yet.

dpkg: error processing package libsdl2-dev:i386 (--configure): dependency problems - leaving unconfigured No apport report written because the error message indicates its a followup error from a previous failure. No apport report written because the error message indicates its a followup error from a previous failure. Errors were encountered while processing: libmirclient-dev:i386 libsdl2-dev:i386 E: Sub-process /usr/bin/dpkg returned an error code (1)

Can someone please help out?

clementgallet commented 5 years ago

Don't install ffmpeg:i386, you don't need it. Maybe it will fix your issue.

qixils commented 5 years ago

when i first tried to build i386 without libswresample i386 (ffmpeg i386) installed it didn't seem to compile on arch. (unless those errors were harmless and i didn't noticed but it seemed to have failed)

InfamousKnight commented 5 years ago

Oh my god..

My exact steps to reproduce: install regular depencies sudo apt-get install build-essential automake pkg-config libx11-dev qtbase5-dev qt5-default libsdl2-dev libxcb1-dev libxcb-keysyms1-dev libxcb-xkb-dev libxcb-cursor-dev libudev-dev libasound2-dev libswresample-dev ffmpeg clone libtas run ./build.sh get 32 bit libraries sudo apt-get install pkg-config:i386 libx11-dev:i386 libsdl2-dev:i386 libudev-dev:i386 libasound2-dev:i386 libswresample-dev:i386

eventually this shows up: Setting up libglib2.0-dev:i386 (2.60.0-1ubuntu0.1) ... dpkg: dependency problems prevent configuration of libmirclient-dev:i386: libmirclient-dev:amd64 (1.1.1-0ubuntu3) breaks libmirclientcpp-dev and is installed. libmirclient-dev:i386 (1.1.1-0ubuntu3) provides libmirclientcpp-dev.

dpkg: error processing package libmirclient-dev:i386 (--configure): dependency problems - leaving unconfigured No apport report written because the error message indicates its a followup error from a previous failure. dpkg: dependency problems prevent configuration of libsdl2-dev:i386: libsdl2-dev:i386 depends on libmirclient-dev; however: Package libmirclient-dev:i386 is not configured yet.

dpkg: error processing package libsdl2-dev:i386 (--configure): dependency problems - leaving unconfigured No apport report written because the error message indicates its a followup error from a previous failure. Processing triggers for install-info (6.5.0.dfsg.1-4build1) ... install-info: warning: no info dir entry in /usr/share/info/gnash_user.info.gz' install-info: warning: no info dir entry in/usr/share/info/gnash_ref.info.gz' Processing triggers for libglib2.0-0:amd64 (2.60.0-1ubuntu0.1) ... Processing triggers for libglib2.0-0:i386 (2.60.0-1ubuntu0.1) ... Processing triggers for libc-bin (2.29-0ubuntu2) ... Processing triggers for man-db (2.8.5-2) ... Setting up libibus-1.0-dev:i386 (1.5.19-1ubuntu2) ... Setting up libpulse-dev:i386 (1:12.2-2ubuntu3) ... Errors were encountered while processing: libmirclient-dev:i386 libsdl2-dev:i386 E: Sub-process /usr/bin/dpkg returned an error code (1)

What are your exact steps?

clementgallet commented 5 years ago

You can try skipping i386 packages and try to build 32-bit libtas, then you will see which package you will miss. I expect most -dev:i386 packages to be unneeded if you have the amd64 package already.

chungy commented 5 years ago

Don't install ffmpeg:i386, you don't need it. Maybe it will fix your issue.

This makes me curious. I created a lib32-libtas for Arch, and I was unable to get it to compile at all without having lib32-ffmpeg. Is there any advice you might have to not requiring it?

clementgallet commented 5 years ago

I guess because Arch ffmpeg contains libswresample.so, which is required, whereas it is bundled in a specific package for Debian-based distributions (and ffmpeg package contains ffmpeg binary)

InfamousKnight commented 5 years ago

This is a nightmare.. Trying to build without 32 bit depencies says nothing but need to install 32 bit development libraries. I look online for a way to list depencies and come across rdepends package, I try to install that and the same error message above comes up. I remove libmirclient-dev:i386 and install rdepends. Using rdepends on build.sh, and it just says unable to locate package build.sh.

Running ubuntu 19.04 64 bit. Should I install arch?

clementgallet commented 5 years ago

Could you paste the output of ./build.sh --enable-i386?

InfamousKnight commented 5 years ago

checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for a thread-safe mkdir -p... /bin/mkdir -p checking for gawk... gawk checking whether make sets $(MAKE)... yes checking whether make supports nested variables... yes checking for g++... g++ checking whether the C++ compiler works... yes checking for C++ compiler default output file name... a.out checking for suffix of executables... checking whether we are cross compiling... no checking for suffix of object files... o checking whether we are using the GNU C++ compiler... yes checking whether g++ accepts -g... yes checking whether make supports the include directive... yes (GNU style) checking dependency style of g++... gcc3 checking build system type... x86_64-pc-linux-gnu checking host system type... x86_64-pc-linux-gnu checking for i386 cross-compiling... no configure: error: Cannot build a 32-bit program, you need to install 32-bit development libraries.

clementgallet commented 5 years ago

Ok, it means your compiler cannot cross-compile (-m32 flag does not work), so you should need g++-multilib package. Edit: sorry, the error message is actually misleading.

InfamousKnight commented 5 years ago

And another issue.

/usr/bin/ld: skipping incompatible /usr/lib/x86_64-linux-gnu/libfreetype.so when searching for -lfreetype /usr/bin/ld: skipping incompatible /usr/lib/x86_64-linux-gnu/libfreetype.a when searching for -lfreetype /usr/bin/ld: cannot find -lfreetype collect2: error: ld returned 1 exit status make[2]: [Makefile:854: libtas32.so] Error 1 make[2]: Leaving directory '/home/seab/Downloads/libTAS-wine/src/library' make[1]: [Makefile:454: all-recursive] Error 1 make[1]: Leaving directory '/home/seab/Downloads/libTAS-wine' make: *** [Makefile:353: all] Error 2

Tried installing libfreetype and lfreetype but they are unknown packages.

clementgallet commented 5 years ago

libfreetype6:i386

InfamousKnight commented 5 years ago

Doesn't make any difference. Apparently it was already installed beforehand. I tried researching the problem online and no luck. Should I reinstall my operating system?

InfamousKnight commented 5 years ago

Okay, I have tried reinstalling operating system for a fresh install and still the same problem.

However, if I install the regular packages and then install the 32 bit packages, qt gets removed. And like all the 64 bit packages are removed I suppose. It does say while installing 32 bit packages xcb packages as well. It does advise that I run autoremove as I don't "need" the 64 bit packages anymore, but I don't run that.

Anyone have any clue what to do here? Running 64 bit ubuntu. Its just we most likely need 32 bit packages for most windows games. I would imagine 64 bit win games run extremely slow if anything(given that they will likely be new).

madewokherd commented 5 years ago

You don't need 32-bit qt.

If a 64-bit game is available, it should run just as well as a 32-bit build of the same game.

Unfortunately, Ubuntu's multi-arch support is incomplete, and many -dev libraries cannot be installed as 32-bit and 64-bit simultaneously. In these cases, I'd suggest leaving the 64-bit libraries installed. Usually all that's required in those cases is a symlink from libwhatever.so.1 (or whatever the version number happens to be) to libwhatever.so.

You could also use a 32-bit vm or chroot to build the 32-bit library.

I'm afraid there's no good way to do this on 64-bit ubuntu.

InfamousKnight commented 5 years ago

Well, thanks for the response. I'm aware 32 qt is not required, I didn't install it or xcb and it still showed up somehow.

Anyways, how do you use chroot for this?

clementgallet commented 5 years ago

Don't install anything :i386 yet and run the build script to see which package does it actually need. And post the error here.

InfamousKnight commented 5 years ago

Um, I ran autoremove to presumably remove :i386 packages and it still compiled like halfway until this:

/usr/bin/ld: skipping incompatible /usr/lib/x86_64-linux-gnu/libX11-xcb.so when searching for -lX11-xcb /usr/bin/ld: skipping incompatible /usr/lib/x86_64-linux-gnu/libX11-xcb.a when searching for -lX11-xcb /usr/bin/ld: cannot find -lX11-xcb collect2: error: ld returned 1 exit status make[2]: [Makefile:854: libtas32.so] Error 1 make[2]: Leaving directory '/home/sean/Downloads/libTAS-master/src/library' make[1]: [Makefile:454: all-recursive] Error 1 make[1]: Leaving directory '/home/sean/Downloads/libTAS-master' make: *** [Makefile:353: all] Error 2

Different from last time at least.

clementgallet commented 5 years ago

Ok, then install libx11-xcb1:i386

InfamousKnight commented 5 years ago

Um, it was already installed beforehand. And it still shows up. Are you using 19.04? Or should I change distro?

InfamousKnight commented 5 years ago

Okay, assuming ubuntu is the problem here, I installed manjaro, and tried to build the 32 bit package(its an arch distro), and eventually this error popped up when build.sh --enable-i386

/usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0/../../../libavutil.so when searching for -lavutil /usr/bin/ld: skipping incompatible /usr/lib/libavutil.so when searching for -lavutil /usr/bin/ld: cannot find -lavutil /usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-pc-linux-gnu/8.3.0/../../../libswresample.so when searching for -lswresample /usr/bin/ld: skipping incompatible /usr/lib/libswresample.so when searching for -lswresample /usr/bin/ld: cannot find -lswresample collect2: error: ld returned 1 exit status make[2]: [Makefile:853: libtas32.so] Error 1 make[2]: Leaving directory '/home/sean/Downloads/libTAS-master/src/library' make[1]: [Makefile:453: all-recursive] Error 1 make[1]: Leaving directory '/home/sean/Downloads/libTAS-master' make: *** [Makefile:352: all] Error 2

I think I'm getting close. What would I need to install?

clementgallet commented 5 years ago

The error message is pretty clear to me, you need libraries libavutil and libswresample

InfamousKnight commented 5 years ago

[sean@sean-pc ~]$ sudo pacman -S libavutil [sudo] password for sean: error: target not found: libavutil

what happened?

On Mon, Jun 17, 2019 at 12:28 PM Clément Gallet notifications@github.com wrote:

The error message is pretty clear to me, you need libraries libavutil and libswresample

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/clementgallet/libTAS/issues/54?email_source=notifications&email_token=AAT4W4L7XXWVDNFELUDJCALP263T3A5CNFSM4FPFFZVKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODX3XFHA#issuecomment-502755996, or mute the thread https://github.com/notifications/unsubscribe-auth/AAT4W4L6O5OXJJLAD6BBBR3P263T3ANCNFSM4FPFFZVA .

chungy commented 5 years ago

I installed manjaro, and tried to build the 32 bit package(its an arch distro), and eventually this error popped up when build.sh --enable-i386

You might try having libtas-git and lib32-libtas-git from the AUR. They should work; if you use a AUR helper like yay, it would make it really easy for all the dependencies to be installed too.

InfamousKnight commented 5 years ago

Welll, thanks. But I am pretty new this kind of stuff.I installed yay, but I don't know what to run now. I tried yay --help and I'm a little confused on what to do.

Thanks for your patience.

On Mon, Jun 17, 2019 at 1:03 PM Mike Swanson notifications@github.com wrote:

I installed manjaro, and tried to build the 32 bit package(its an arch distro), and eventually this error popped up when build.sh --enable-i386

You might try having libtas-git https://aur.archlinux.org/packages/libtas-git/ and lib32-libtas-git https://aur.archlinux.org/packages/lib32-libtas-git/ from the AUR. They should work; if you use a AUR helper like yay, it would make it really easy for all the dependencies to be installed too.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/clementgallet/libTAS/issues/54?email_source=notifications&email_token=AAT4W4ONCP5VAGXMEZC2IGTP267X5A5CNFSM4FPFFZVKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODX32HUY#issuecomment-502768595, or mute the thread https://github.com/notifications/unsubscribe-auth/AAT4W4N5QVFK6PPJLHOOLSTP267X5ANCNFSM4FPFFZVA .

madewokherd commented 5 years ago

This may have gotten lost in the thread: the reason for building from source was to use the wine branch rather than the master branch. Is this possible with the -git packages?