exult / exult

Exult is a project to recreate Ultima 7 for modern operating systems, using the game's original plot, data, and graphics files.
http://exult.info
GNU General Public License v2.0
515 stars 82 forks source link

Porting to SDL3 #379

Open DominusExult opened 1 year ago

DominusExult commented 1 year ago

SDL2 has now gone into maintenance mode and massive work is being done on SDL3 (on SDL's github main is now SDL3 and SDL2 is branched). I suggest that this time we switch faster than with SDL2 by reviving the old configure option --with-sdl= and sdl-compat.h (probably best, if possible, to rewrite functions to their SDL3 name and redefine for SDL2 in sdl-compat.h?). Faster to have less work in the future than if we get compatible sooner.

One of the goals of SDL3, if I'm not mistaken or things changed, is to add https://github.com/icculus/mojoshader or similar to SDL3 to allow one shader for all platforms. Which I would love to see :)

Maybe @Dragon-Baroque is interested in porting :)

https://github.com/libsdl-org/SDL/blob/main/docs/README-migration.md

Edit: looking at the migration doc, it seems there would be a lot of ifdefs...

DominusExult commented 1 year ago

and the plans (or rather brand new implementation I think) for the audio in SDL3 https://www.patreon.com/posts/sdl3-audio-87089007 (might be interesting for @wench as you did a lot on our original audio back then).

wench commented 1 year ago

Audio wise, I'll check out what's new and if there will be any big gotchas and breaking changes. Guessing Exult itself probably wont need much in the way of changes. Graphics wise, I have been dragging my feet on making big changes because I don't really want to change Exult's behaviour from a user point of view but if SDL3 requires changes, then that might be an opportunity to change things for the better. Using the Accelerated SDL graphics routines would be nice but Exult would need a lot of changes for that. Exult really does feel like 20 year old software at times.

Dragon-Baroque commented 1 year ago

First assessment :

Dragon-Baroque commented 1 year ago

SDL3 mouse events become floats because they can be sub-pixel defined. What should we consider :

DominusExult commented 1 year ago

In the long run it seems to me that float is where the SDL guys want to go. So maybe run along with them? I think the display portion moves that way, too (with Hidpi being always on...). But you guys know more ;)

rofl0r commented 1 year ago

Move all pixel definitions in Exult from int to float, or

that seems like a horrible idea for performance, most cpus still have much faster integer engines, especially when it comes to loops. i'd just abandon the decimal part of the mouse events or round them.

wench commented 1 year ago

Floats for fractional coordinates are nice and useful in 3d games where you are working with triangles and off axis lines. Exult is purely 2d with everything on the pixel grid so we have no use for floats. You just end up with the float comparison problem with no advantages. If we needed fractional components i'd prefer fixed point.

Dragon-Baroque commented 1 year ago

@rofl0r, you are true, though :

@wench, Exult is purely 2D, true, but so is SDL, correct ? Then what is the purpose of introducing fractional pixels in SDL ? Could that be related to HiDPI ?

marzojr commented 1 year ago

XMM registers are SSE registers; original SSE assumed they were always operating on 4 x 32-bit IEEE floats, but SSE2 and above (first seen in Pentium 4) can also operate on XMM registers that have 16 x 8-bit, 8 x 16-bit, 4 x 32-bit, or 2 x 64-bit integers, or 2 x 64-bit floats.

And if I am not mistaken, SDL supports 3D graphics as well. But even in a 2D game you may want to use something other than integers: many older games (especially platformers) used something like 16.4, 16.8, or 16.16 fixed point formats for position so that they could have non-pixel speeds, which are too coarse-grained for smooth motion.

Of course, having subpixel position in Exult would probably not matter much as we have speeds in pixels (with smooth scrolling on) or tiles (if not).

Dragon-Baroque commented 1 year ago

I have a prototype port of Exult to SDL3.

SDL3 contains a massive rework of the Audio part. At this moment, my port uses the SDL3 audio exactly like the SDL2 audio, using a callback to feed sample audio data to SDL3. But the preferred way is to provide the whole song at once as a SDL_AudioStream and let SDL3 schedule ( and convert and resample ) the audio data to the audio device. It also can bind several SDL_AudioStream to the audio device, making, I believe, the Exult audio mixer irrelevant.

Now for me to test that, could you identify a place in BG or SI where several audio streams occur at the same time ? Is it the case for example at the beginning of BG, with the music sounding while the Avatar crosses the red gate ( and makes sound effects thereby ) ?

By the way, nagging me is the question : The Audio gump has a Music -> Looping attribute. When does it take effect ? I mean, for me the same initial music of BG does not loop ...

wench commented 1 year ago

The looping flag from memory would disable the infinite looping of music eggs. It does not force enable looping of eggs set to not loop as that tends to leave certain songs playing forever

Multiple streams probably would only occur with fmopl music and sfx\voice playback and multiple sfx at the same time. Sfx and voice playback could be handed over entirely to SDL, fmopl and integrated software midi synths were architectured around the callback method and changing them might be a lot of work

DominusExult commented 1 year ago

Wheee! Thank you!

I think the best place to check for multiple audio streams is the BG intro. There you have music, sfx (with the sfx packs) and speech.

wench commented 1 year ago

Completely forgot about ogg music being another audio source that we mix.

Dragon-Baroque commented 1 year ago

Thank you @wench, and what about :

wench commented 1 year ago

Hardware midi drivers are multithreaded they run independently of the rest of the audio system they do not produce samples and have no interaction with the callback.

Ogg music seems like a matter of how exactly SDL intends for streaming compressed music to work. The callback is very convenient. as we can just in time produce samples at exactly the correct rate.

Dragon-Baroque commented 1 year ago

I think the best place to check for multiple audio streams is the BG intro.

Thank you, good test indeed. The port seems to work OK at this point, I mean using a SDL2 style with the Audio callback and a single SDL_AudioStream.

The callback is very convenient. as we can just in time produce samples at exactly the correct rate.

The way SDL3 is written, the callback is there to stay, I believe. I have made a few experiments :

Dragon-Baroque commented 1 year ago

Protoype SDL3 for Exult

I have pushed a branch exult-sdl3 onto my fork Dragon-Baroque/Exult.

It contains a prototype for Exult migrated to SDL 3. It works on my Fedora Linux.

The branch actually contains two commits. I put into the first commit the automatic changes made by the SDL 3 provided rename_symbols.py, along with some cosmetic changes. I put into the second commit the real migration effort, that is :

To be completed SDL 3 Audio now uses SDL_AudioStream that schedule / convert audio data to the audio device, you do not reach the SDL_AudioDevice directly. This impacts the audio callback architecture.

SDL 3 Video in Full Screen only provides the max resolution of the display. SDL3 handles all actual resolutions by scaling. This impacts the VideoOptions gump.

I have not tested the JoyStick / GamePad support. Could someone tell me how to ?

Installation of SDL 3 Download the Git tree of SDL : git clone https://github.com/libsdl-org/SDL.git SDL.git The build and installation of SDL 3 are described in docs/README-porting.md. It uses cmake. For Linux and macOS, and for Windows MingW with some option to the first cmake command to locate the compiler, from SDL.git :

Once installed, SDL 3 is pkg-config compliant, and does not interfere with SDL 2 ( it is named libSDL3.so ).

DominusExult commented 1 year ago

Oooh! Looking forward to test this beginning of next week. Especially how the iOS and Android ports will behave.

To test the gamepad/joystick code, you just need the hardware and test if you can move the party.

If SDL3 graphics code only does the full fullscreen res, we might need to set an array of resolutions to choose from?

Dragon-Baroque commented 1 year ago

To test the gamepad/joystick code, you just need the hardware and test if you can move the party.

Fine. I checked on master ( SDL 2 ) to practice. Then on exult-sdl3 ( SDL 3 ) to compare. Using a standard gamepad (?) : two joysticks, one cross, four buttons in diamond, four buttons in front. With both, I moved the Avatar and Iolo ( beginning of Black Gate ) with the left joystick. How does one click ( conversation ) or act on objects ( open door ... ) ?

If SDL3 graphics code only does the full fullscreen res, we might need to set an array of resolutions to choose from ?

This seems to be the case for windowed Exult. The hardcoded windowed resolutions came first in Exult by 2010, commit 2efd0af7. We might reuse the windowed array of resolutions for fullscreen. On the other hand, the present array stops at 1920x1440 and is made of 4:3 and 16:10 combinations. This means that my 16:9 full hd does not fit, and would that be acceptable to a larger screen ?

DominusExult commented 1 year ago

The joystick code ONLY moves the party. Historically this was only implemented for the virtual gamepad on iOS (and later made use of by Android), on which you need to be anle to move the party but everything else is done by using your finger. @marzojr has plans to implement more but the current movement code being all over the codebase extremely irked him.

Resolution: Probably adding more resolutions is desireable, I'd say. And then we would also need to cut the list short if it exceeds the fullscreen limits of the display.

DominusExult commented 1 year ago

@Dragon-Baroque: trying to build the iOS port I've run into these problems:

/imagewin/save_screenshot.cc:209:4 No matching function for call to 'SDL_RWwrite'
/imagewin/save_screenshot.cc:212:4 No matching function for call to 'SDL_RWwrite'
/imagewin/save_screenshot.cc:213:4 No matching function for call to 'SDL_RWwrite'
/imagewin/save_screenshot.cc:290:2 No matching function for call to 'SDL_RWwrite'
/imagewin/save_screenshot.cc:297:3 No matching function for call to 'SDL_RWwrite'
/imagewin/save_screenshot.cc:298:3 No matching function for call to 'SDL_RWwrite'
/imagewin/save_screenshot.cc:303:4 No matching function for call to 'SDL_RWwrite'
/imagewin/save_screenshot.cc:304:4 No matching function for call to 'SDL_RWwrite'
/imagewin/save_screenshot.cc:305:4 No matching function for call to 'SDL_RWwrite'

/ios/GamePadView.m:76 'SDL_NumJoysticks' had to be renamed to 'SDL_GetJoysticks' but that leads to /ios/GamePadView.m:76:40 Too few arguments to function call, single argument 'count' was not specified This needs to be done as in exult.cc (but over my head). Follow up problem is /ios/GamePadView.m:77:38 Call to undeclared function 'SDL_JoystickGetDeviceInstanceID'; as SDL_JoystickGetDeviceInstanceID was removed in SDL3.

-> Does this make sense?:

- (void)dealloc {
    if (vjoy_controller) {
        const SDL_JoystickID *vjoy_controller_id = SDL_GetJoysticks(0);
        SDL_CloseGamepad(vjoy_controller);
        for (int i = 0; vjoy_controller_id[i]; ++i) {
            if (SDL_IsGamepad(vjoy_controller_id[i])) {
                // printf("detach virtual at id:%d, index:%d\n", current_id, i);
                SDL_DetachVirtualJoystick(i);
                break;
            }
        }
    }

and finally: exult.cc:2470:7 No matching function for call to 'SDL_GetDesktopDisplayMode' in

        SDL_DisplayMode dispmode;
        if (SDL_GetDesktopDisplayMode(0, &dispmode) == 0) {
            w = dispmode.w;
            h = dispmode.h;
        }

<- I have no idea anymore what that does, but it's likely from me :)

Does this change make sense? (it compiles at least)

        const SDL_DisplayMode * dispmode = SDL_GetDesktopDisplayMode(SDL_GetPrimaryDisplay());
        if (dispmode == 0) {
            w = dispmode -> w;
            h = dispmode -> h;
        }
DominusExult commented 1 year ago

and iOs needs this little patch for Xcode finding the SDL3 includes (and grabbing the SDL git). Reason I used iOs first was because it doesn't need me to build SDL3 and tinker with everything and Xcode spits out nice fails/warnings out. sdl3.patch

After clobbering and commenting stuff in /ios/GamePadView.m and /imagewin/save_screenshot.cc (and what relied on it), Exult almost compiled and ran but needed #include <SDL_main.h> in exult.cc

Dragon-Baroque commented 1 year ago

@DominusExult

I have pushed a new commit to the branch exult-sdl3.

DominusExult commented 1 year ago

I've edited my two posts (before your answer) with additional info and proposals that seem to work somewhat and further stuff needed (SDL_main.h). I need to figure more out. My touch input is wrong, my guess is that I was making changes on whether hidpi was enabled, but it is now always enabled in SDL3 and thus my inputs are wrong :)

Dragon-Baroque commented 1 year ago

I read your edits only afterwards. At least you should invert the dispmode test : if ( dispmode != nullptr ) ( nullptr better than 0 ) instead of if ( dispmode == 0 ).

Addon Looking at test/testcontroller.c in SDL 3 compared to SDL 2 also gives me some ideas on how to alter the Virtual Joystick code too. Some more news soon ...

Addon Commit force pushed to exult-sdl3 with new test over dispmode in exult.cc as well as code rework of Virtual Joystick setup and removal for both iOS and Android. Unable to test and even to compile, though.

DominusExult commented 1 year ago

could you add this patch to your branch? It's necessary xcode changes which are annoying to apply (header search path, removing fluidsynth as it has no SDL3 port, removing SDL2 and adding SDL3) and a slightly bigger change that requires you to rename GamePadView.m to GamePadView.mm (or nullptr is undeclared as *m is compiled as C not C++ code :)) xcode2.patch

Another patch is for the on screen ESC button that stopped working because SDL_SendKeyboardKey needs three arguments now (but it seems this is not documented yet and compiling didn't break there either). ios_utils.patch

Still have no valid mouse (or rather finger) input, it doesn't update and always clicks on one point. Starting out with the the upper left corner but once I hit the ESC button (lower left corner it updates somewhat but gets stuck there again. Something wrong with fastmouse?

And finally some warnings:

audio/Audio.cc:85:11 Format specifies type 'long' but the argument has type 'unsigned long long'
audio/Audio.cc:85:25 Format specifies type 'long' but the argument has type 'unsigned long long'
audio/AudioChannel.cc:66:11 Format specifies type 'long' but the argument has type 'unsigned long long'
audio/AudioChannel.cc:66:25 Format specifies type 'long' but the argument has type 'unsigned long long'
audio/Midi.cc:62:11 Format specifies type 'long' but the argument has type 'unsigned long long'
audio/Midi.cc:62:25 Format specifies type 'long' but the argument has type 'unsigned long long'
Dragon-Baroque commented 1 year ago

xcode2.patch

I had forgotten that Objective C was C not C++. My intent was not to force you to move the iOS code to C++. Thus I replaced the offending nullptr by a NULL. I have tried to keep your changes in xcode2.patch except for the no longer needed renaming of GamePadView.m to GamePadView.mm.

ios_utils.patch

It is in the branch now. I have now 4 commits in that branch

By the way, since you are still facing mouse / finger input problems, have you considered validating the SDL 3 port to macOS ?

DominusExult commented 1 year ago

Thank you. I'm just now trying my hands with macOS and SDL3. First hurdle is yesterday's SDL commit 58c859f64 -> audio: Rename SDL_GetAudioStreamBinding to SDL_GetAudioStreamDevice which of course makes your audio code migration fail :( (going with SDL before that commit now and this compiles and runs).

Mouse and screen has some issues but not as severe as the iOS port. Windowed mode works fine.

Fullscreen with SDL2: the input option "fullscreen fastmouse" does not work the pointer is stuck on the lower right corner and won't move. Disabled everything works fine and the mouse pointer is not slow.

Fullscreen with SDL3:

By some glitch the enabled fastmouse did work once, no system cursor visible and I was able to click everywhere, only the teleport to cursor cheat did not work correctly.

This is all an HighDPI issue apparent on iOS and macOS as these both have have these screens and SDL3 only uses this.

DominusExult commented 1 year ago

Seems this thread might give the relevant information on HighDPI https://github.com/libsdl-org/SDL/issues/7134 - no idea if I get it :)

wench commented 1 year ago

If we need to set an array of fullscreen resolutions because SDL no longer has the functionality to query support we might want to add our own platform specific code for listing supported fullscreen modes. Of course if SDL forces scaling in all fullscreen resolutions and doesn't actually change modes this would be kinda pointless

DominusExult commented 1 year ago

Hmm, you are correct, we are not doing exactly that with our game screen resolution anyway. So adding resolutions to our windowed mode makes sense, fullscreen not so much.

DominusExult commented 1 year ago

Seems this thread might give the relevant information on HighDPI libsdl-org/SDL#7134 - no idea if I get it :)

on macOS I was able to fix everything by either discarding the whole if (fullscreen) block in https://github.com/exult/exult/blob/master/imagewin/imagewin.cc#L604 or only letting SDL_SetRenderLogicalPresentation(...) stand. <- something for you to test. Seems the nativescale parts are no longer needed at all. This didn't fix iOS, so I need to further investigate.

DominusExult commented 1 year ago

The iOS problem is caused by an SDL3 bug https://github.com/libsdl-org/SDL/issues/8176

Edit: bug is fixed now

Dragon-Baroque commented 1 year ago

Pushed a new commit to exult-sdl3 to fix Audio API changes as of SDL commit 2471d8 of August 27.

DominusExult commented 1 year ago

Thank you! Need to test this now :) (still am on a heavily debug couted copy)

I fixed the highdpi issues. Mostly it needed more cases of screen_to_game_hdpi instead of screen_to_game only. The fast mouse fix in imagewin.h I should actually apply to our master. In imagewin.cc the nativescale factor is now returned by an SDL function instead of having to be calculated.

highdpi.patch

So apart from the things you left me in the comments to try out, I consider the macOS migration complete.

Next step I'm trying out is the Android port.

Dragon-Baroque commented 1 year ago

highdpi.patch

Would you like me to place it into the exult-sdl3 branch ?

And aren't there quite some more screen_to_game that the patch does not migrate to screen_to_game_hdpi ? What is the difference ?

DominusExult commented 1 year ago

Yes, please add this to the branch.

There are a few places that I haven't changed screen_to_game:

Dragon-Baroque commented 1 year ago

highdpi.patch

Commit on exult-sdl3 branch pushed.

DominusExult commented 1 year ago

oh, I forgot to do more, the Highdpi cfg setting and video options gump setting for it can go.

Edit: or rather I think I can still make use of it. Let me tinker :)

DominusExult commented 1 year ago

@Dragon-Baroque can you add # include <SDL_main.h> to exult.cc's first #ifdef __IOS__ this is needed for SDL3.

And some patches: togglehdpi.patch <- makes somewhat use of the existing high_dpi config setting.

android.patch Android patch. With these changes Exult for Android compiles and runs for a moment. I'm worried about the change I needed to do in android/app/src/main/cpp/exult_android_main.cc:34, commenting auto result = SDL_main(argc, argv); this is probably wrong but without that change I got this compile error: /android/app/src/main/cpp/exult_android_main.cc:34: error: undefined reference to 'SDL_main' And I don't understand much about the changes to SDL_Main stated in https://github.com/libsdl-org/SDL/blob/main/docs/README-migration.md

Anyway, this leads to a crash of Exult right from the start. Could very well be an SDL3 problem and not actually caused by us. Any ideas on this @Dragon-Baroque or @ceckak (sorry for tagging you, but maybe you have an idea right away)?

The relevant parts of the Android emulator logs that I could find were not very conclusive (to me), either: AndroidCrash.txt

The patch is still needed, please add this to your branch (and feel free to change the SDL_main thing if you have an idea).

Dragon-Baroque commented 1 year ago

Commit added to exult-sdl3 with togglehdpi.patch and android.patch and #include <SDL_main.h> in exult.cc.

You are using #include <SDL3/SDL...> whereas my patches are standardized to #include <SDL...> ( without SDL3/ ). This is because using the pkg-config --cflags sdl3 should give you the include paths up to and including SDL3/.

In Exult, branch master, there are no longer any #include <SDL2/SDL...> except for the two files for Android android/app/src/main/cpp/exult_android_main.cc + android_log_streambuf.cc. In the exult-sdl3 branch, my first commit 71b94e1f removed these two explicit SDL2/. Your patch android.patch reintroduces explicit SDL3/ in these two files plus TouchUI_Android.cc. Would you like to check Android without these SDL3/ ?

DominusExult commented 1 year ago

I tried without the SDL3/ and it didn't find the files. Our buildsystem only does the bare minimum before the Android build takes over via gradle, cmake, ninja. So we don't even need an installed SDL3 as the Android buildsystem just grabs the SDL git and builds it while building. Why it is okay with all other SDL includes outside of the Android folder is a mystery to me, though ;)

Dragon-Baroque commented 1 year ago

I would like to know what you mean by runs for a moment.

Because as I understand it, SDL_main is Exult's renamed main, and not calling it means that do not run Exult at all. Now why you get this undefined reference to SDL_main is also a mystery because SDL_main is explicitly declared in SDL_main.h, line 156. If there has been a rename mismatch, you would get a later link error, but not a compile error.

ceckak commented 1 year ago

I'm traveling this week and won't have a chance to dig into this in any detail. From a quick skim through the README you linked, it does sound related to including the correct SDL_main.h, but I haven't done any experimentation with SDL3 yet. Will try to take a look next week.

Ken

DominusExult commented 1 year ago

I would like to know what you mean by runs for a moment.

It seems to flash for a moment but that could just be the android app switcher.

Dragon-Baroque commented 1 year ago

Please confirm me, the error message about SDL_main is coming at link, not at compile as I thought first, yes ?

If yes, then, please add #include <SDL_main.h> in exult.cc also in the #elif defined(ANDROID) branch, since as of now it is only in the #ifdef __IOS__ branch.

Why it was working in SDL2 is that SDL_main.h was included by SDL.h, at the top of exult.cc. No longer in SDL 3. This should affect Linux too, I have to investigate on my side...

Add-On No it does not affect Linux or macOS, but it definitely affects Windows, iOS, Android. The best solution, if adding #include <SDL_main.h> to Android in exult.cc works, would be to put an unconditional #include <SDL_main.h> just below the #include <SDL.h>.

DominusExult commented 1 year ago

I can confirm that this was during linking. I'm right now building for Android with this change. Will only take a long while :)

ceckak commented 1 year ago

As far as the crash at runtime, you will probably need to dig into adb logcat output. My guess is that you're missing a .so in the apk or getting some other kind of runtime link error

DominusExult commented 1 year ago

@Dragon-Baroque @ceckak: success! in android/app/src/main/cpp/exult_android_main.cc:34, uncommented my change to auto result = SDL_main(argc, argv); and added #include <SDL_main.h> to Android inexult.cc. It runs and some stuff works. Unfortunately the same SDL3 bug concerning the mouse input strikes here as well. Even worse as the first touch renders Exult unresponsive for some reason. But that is for the SDL3 devs to fix I guess.

Adding #include <SDL_main.h> unconditionally does not have an ill effect on compiling and running Exult on macOS.

Dragon-Baroque commented 1 year ago

Latest commit on exult-sdl3 :