Updated ScummVM core based on latest sources available if you want it

[diablodiab]

I've made a new updated version of the core based on the existing libretro wrapper and the newest ScummVM sources. There are also some adjustments in the approach that completely separates the wrapper from the original source, hopefully making it easier to keep the core in sync going forward.

If this is of interest you can find it here: https://github.com/diablodiab/scummvm

The libretro-wrapper is added in one commit on top of the base source.

You are more than welcome to incorporate or use it for inspiration towards the official core if you want to. Just wanted to share.

[hizzlekizzle]

oh, that's very exciting, actually! Thanks for doing this work.

[dhwz]

@diablodiab Can you tell me how to build it properly for aarch64? I always get an compile error.

make: *** [Makefile:559: engines/ags/engine/ac/characterinfo_engine.o] Error 1

[diablodiab]

You can try commenting out "ENABLE_AGS" line in config.hd to see if it might be an issue with the AGS engine for the platform you are compiling for. Out of curiosity, what device are you trying to compile it for?

[dhwz]

Oh yes commenting out helped great thx! I tried to disable it by setting it to 0 but that didn't help. :-)

I'm building it for 351ELEC (Anbernic RG351).

[dhwz]

@diablodiab Another question maybe you have an idea, saving core overrides doesn't work (with your version and also the old version). "Error saving overrides"

[diablodiab]

I'm not sure this is related to the core or to RetroArch itself. I get the same error with many other cores at the moment.

[dhwz]

Yes that was also my guess that it is RetroArch related but some user doesn't believe me.

[DrUm78]

Amazing work, thanks! I compiled it successfully for the FunKey S (after disabling AGS that failed too). :)

[DrUm78]

@diablodiab Here is the error message I get when trying to compile the Libretro core with your wrapper with last sources from ScummVM repo (https://github.com/DrUm78/scummvm/commit/4aa78f28b3610ff1875a99e9bbe7a417ee4b723a):

Compiling libretro_os.cpp...
../../../../backends/platform/libretro/libretro_os.cpp: In function ‘OSystem* retroBuildOS(bool)’:
../../../../backends/platform/libretro/libretro_os.cpp:1271:45: error: invalid new-expression of abstract class type ‘OSystem_RETRO’
 1271 |    return new OSystem_RETRO(aEnableSpeedHack);
      |                                             ^
../../../../backends/platform/libretro/libretro_os.cpp:288:7: note:   because the following virtual functions are pure within ‘OSystem_RETRO’:
  288 | class OSystem_RETRO : public EventsBaseBackend, public PaletteManager {
      |       ^~~~~~~~~~~~~
In file included from ../../../../backends/base-backend.h:26,
                 from ../../../../backends/platform/libretro/libretro_os.cpp:33:
../../../../common/system.h:1369:15: note:     ‘virtual void OSystem::getTimeAndDate(TimeDate&, bool) const’
 1369 |  virtual void getTimeAndDate(TimeDate &td, bool skipRecord = false) const = 0;
      |               ^~~~~~~~~~~~~~
make: *** [Makefile:564: ../../../../backends/platform/libretro/libretro_os.o] Error 1

The last working commit for me is https://github.com/DrUm78/scummvm/commit/10310cccdd7f8052515d3d3f181a94a21992497d. Seems like the ScummVM team changed something in the code that does not work anymore with your wrapper. My repo: https://github.com/DrUm78/scummvm

Any way to fix it on your side? Thanks!

[diablodiab]

I've merged with base a couple of times since the first post here. I think I fixed this specific issue in this commit on my repo: https://github.com/diablodiab/scummvm/commit/b4064be09dcf98afe757d97102aa64b9be6f0e1a

[diablodiab]

Nvm, I see changes have been made yesterday that causes your the issue you are having. You need to apply this update to the getTimeAndDate function libretro_os.cpp: https://github.com/scummvm/scummvm/commit/a0c237f7b933b444e7fa297e090266a92aca45a8#diff-736a7d91012aa31c738fe94bc53f4443daf2a2f6bee4aa623c8ca2c73360e50f

[DrUm78]

Amazing, that works, thank you for your very quick help and your great work. 🥇

[mrmatteastwood]

Nothing to contribute here, but just wanted to say: freaking brilliant. I really hope your work makes it into the official RA repos, it would be amazing to have an up-to-date ScummVM core!

[hizzlekizzle]

I compiled this and did some admittedly brief testing, and everything that I tried seemed great. I did have to disable AGS (linux x86_64) to get it to compile, but that was the only hitch.

The current scummvm core is newer than I thought (it was apparently updated circa 2018 vs the 2011 I was thinking), but the git history is completely buggered, so I think switching over to this would be a good idea anyway.

[mrmatteastwood]

Would one of you fine folks be willing to share a build of this for Linux? I'm no good at building stuff myself, but I'll be more than happy to do testing.

[DrUm78]

@hunterk Your file was deleted it seems.

[hunterk]

weird. oh well. deleted the post, too /shrug

[DrUm78]

I deleted mine too as it has a few perks for my targetted device (240x240 resolution and some other changes) so that won't apply for a generic Linux distro.

[DrUm78]

@mrmatteastwood Here is the 2.3.0 core for Linux from last diablodiab's sources (sources are 3 days old only) as my repo has too many changes for another platform: https://www.mediafire.com/file/gov4s3a1ld0deqo/scummvm_libretro.zip/file

I did not test it but should work as it does for my ARM device. Binary information is: scummvm_libretro.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=bfaea00c56b3f3c4817413e5f972a7ee0225aeb8, stripped

It has all the engines, except AGS (because of the compilation error). Enjoy.

[diablodiab]

I don't use Linux that much myself, but I've also compiled a Linux version of the core now: http://lindqvist.synology.me/linux/x86_64/scummvm_libretro.so.zip

This one includes the AGS engine also.

[mrmatteastwood]

I don't use Linux that much myself, but I've also compiled a Linux version of the core now: http://lindqvist.synology.me/linux/x86_64/scummvm_libretro.so.zip

This one includes the AGS engine also.

Thanks so much for your replies, guys! I am going to use this one and will test it with the games I have over the coming days, running RA 1.6.0 on Linux Mint 20.2.

[mrmatteastwood]

Hey @diablodiab, is it possible to report bugs directly in your GitHub repo? I don't see an "Issues" tab. If not, I can report the issues here.

[diablodiab]

I've enabled issues on my repo now, just to keep things a little separated for now.

[mrmatteastwood]

I found 2 bugs so far, though I have to say one seems quite specific, so not sure how much you can do about them. Either way, perhaps this helps a little to refine your new version of this core.

That said, I tried a number of other games, including the Gobliiins series, some Lucas Arts adventures, Broken Sword. Didn't play them very far, but so far, no issues, all seems stable and reliable. The animations in Broken Sword felt a little slow and the mouse cursor felt a little laggy, but I seem to recall that's how this game always was, so I don't think it's a bug.

[i30817]

It's not surprising, new engines don't often change the rest of the code nowadays.

I think that upstream is not up to date in having all the checksums to all AGS < 3.5 games yet though (and may never will since they're thousands even not including demos, patches and incomplete 'games').

Oh and i wanted to say 'thank you' for the linux ags enabled core posted here which i locked in my retroarch to keep it until this is merged.

AGS 3.5 which scummvm ported is a moderately good place with only some major games with minor problems in games of the Kathy Rain studio currently. Only major thing left besides that is getting rid of Allegro and keeping MIDI emulation with samples working and one or two minor games that use the lua plugin (i'd call them demos even), that i think might not happen now since AGS 4.0 is breaking away with compatibility anyway, so if it happens it'll happen on the scummvm repo i think.

[mrmatteastwood]

It's not surprising, new engines don't often change the rest of the code nowadays.

I was thinking the same thing. Glad you went into some more depth and details, this is more than I can do at the moment. You're experiencing the same issue with distorted music when using sound fonts?

[i30817]

I haven't tested ags in the core that much.

The problem it had with sound fonts upstream until recently i was writing about was that Allegro controlled the soundfont location with a config file or 'fell back' to the executable directory or the 'home' directory.

This worked fine in windows, because the 'executable directory' is the same as the game data. It didn't work in linux because people won't ever write config files for libraries, much less libraries they didn't install and are statically linked, and the executable was not in the game dir (libretro core isn't either), and so people were forced to put in the soundfonts in the home dir to have functioning MIDI if they hadn't setup timidity and configured things just right. If a MIDI game came with a soundfont - which is rare because they add 15 mb to the download - they'd never have music and would never know better if they hadn't figured this out (which took me a while).

I'm talking about the 'emulation' allegro midi driver using the soundfont; not sending the MIDI data directly to the soundcard, which frankly speaking never worked right to me. I heavily suspect nearly all soundcards in windows use midi emulation now because that shit hasn't been working in linux without a weird timidity or fluidsynth system configuration almost no one does for literal decades, so having the option to configure AGS to use the allegro MIDI emulator directly (probably a old static version of timidity) was a configuration blessing to make the games move across computers with a minimum of breakage.

This location bug was fixed upstream though, now the engine will pickup soundfont files in the same directory as the game data files.

I did notice a bug where the fonts are broken (transparency artifacts) in several games, which i suppose is due to scummvm ripping out whatever library the original engine uses for font handling for theirs. This is going to be a 'problem' for more stylized games even after that is fixed i think.

[i30817]

Yo, I don't know if you're interested in it, but early macromedia games just got added to the main build upstream. This is not that interesting yet, but cosmology of kyoto and frankenstein through the eyes of the monster may be supported soon.

[diablodiab]

Merged latest ScummVM sources today, enabled a few engines that were not previously enabled (SAGA2 and Trecision), and compiled the core for a number of systems.

Windows x86_64: http://lindqvist.synology.me/windows/scummvm_libretro.dll.zip

Linux x86_64: http://lindqvist.synology.me/linux/x86_64/scummvm_libretro.so.zip

MacOS arm64: http://lindqvist.synology.me/applem1/scummvm_libretro.dylib.zip

Android armeabi-v7a: http://lindqvist.synology.me/shield/scummvm_libretro_android.so.zip

Android arm64-v8a: http://lindqvist.synology.me/shield64/scummvm_libretro_android.so.zip

[mrmatteastwood]

This is awesome! Let's hope the RA team considers your work for inclusion into the official repos soon.

EDIT: just checked, bug https://github.com/diablodiab/scummvm/issues/1 still occurs.

[i30817]

Do those builds include AGS and macromedia director engines?

[Darknior]

WOAW so impressive and so exciting :) If you have the latest source code with the AGS Engine it will be fantastic !!!

[diablodiab]

Yes, those builds include AGS and Director engines

[illando]

@diablodiab please would you compile also a MacIntel version making it available for downloading?

thanks a lot!

[diablodiab]

I'm assuming you mean Intel-based Mac (there is already one for M1 above). Here you go.

MacOS x86_64: http://lindqvist.synology.me/appleintel/scummvm_libretro.dylib.zip

[DrUm78]

@diablodiab Compilation is broken again with the last source from ScummVM repo:

Compiling libretro-fs.cpp...
In file included from ../../../../backends/platform/libretro/libretro-fs.cpp:31:
../../../../backends/platform/libretro/libretro-fs.h:92:25: error: invalid covariant return type for ‘virtual Common::String LibRetroFilesystemNode::getDisplayName() const’
   92 |  virtual Common::String getDisplayName() const { return _displayName; }
      |                         ^~~~~~~~~~~~~~
In file included from ../../../../backends/platform/libretro/libretro-fs.h:26,
                 from ../../../../backends/platform/libretro/libretro-fs.cpp:31:
../../../../backends/fs/abstract-fs.h:127:28: note: overridden function is ‘virtual Common::U32String AbstractFSNode::getDisplayName() const’
  127 |  virtual Common::U32String getDisplayName() const = 0;
      |                            ^~~~~~~~~~~~~~
make: *** [Makefile:568: ../../../../backends/platform/libretro/libretro-fs.o] Error 1

[diablodiab]

@DrUm78 They seem to be making many adjustments to types in base these days. I'm not sure if it makes sense to merge with base often or make wait a while and deal with all adjustments at once.

If you see these types of compilation errors after merging, it is usually quite easy to figure out which adjustments to make to the libretro backend. This error shows they've made type changes to the getDisplayName in backends/fs/abstract-fs.h, so if you check recent changes to this file in base you will see this commit: https://github.com/scummvm/scummvm/commit/8c863199232ca9fe80bfba4569e018a0b091f84f#diff-b77e756fb7bb2764d0f283de8b5001b8d99e230dbeeb162dda3a319140b3b492

You need to adjust accordingly in libretro-fs.h:92 - change type from Common::String to Common::U32String

Same way to fix issues when they change other things later.

[DrUm78]

Yes I will have a look at that, best way to progress. And you're right, that's certainly better to wait a bit to "fix" all those adjustments in the Libretro wrapper. Thanks again for your work, which allowed me to easily port ScummVM on the FunKey S with the last source. 👍

[illando]

I'm assuming you mean Intel-based Mac (there is already one for M1 above). Here you go.

MacOS x86_64: http://lindqvist.synology.me/appleintel/scummvm_libretro.dylib.zip

Thanks a lot @diablodiab: I tried 2 Residual games that can finally run using this updated version of the core:

• Grim Fandango
• Escape form Monkey Island

The graphic seems fine, but compared to the standalone ScummVM version for mac, there are some sound problems, especially in Escape form Monkey Island.

[diablodiab]

Yes, I think this is mainly due to TInyGL which is used as an software-based alternative to real OpenGL in the libretro-core. Grim seems to run better while EMI has sound stutter (mainly in cut scenes).

Tapping into real hardware accelerated 3D available in Retroarch for games that make use of OpenGL is something for the libretro guys to look into at some point.

[ckeisc]

Really great to see better alternative to Scummvm core in Retroarch. @diablodiab - have you thought about enabling UWP build? There is UWP support at least in the current scummvm core in libretro repo, but the core itself has some VFS related issues that never quite works on an Xbox. There is a vibrant community of Retroarch users on Xbox hoping for a functional and uptodate scummvm core (myself included of course).

[ToniBC]

There are several cores that are very outdated, such as Beetle Saturn or erratic performance like Dolphin, I imagine due to lack of update or revision.

In this one they go for version 2.5.0 with very interesting additions, but they left it somewhat abandoned.

It is not necessary that it be updated every day, but they would have to go through several cores to update them, which, if they spend 2 months updating them, nothing happens, but it is not just creating the core, it is maintaining it. They would have to find someone for those tasks, maintenance.

Hopefully things progress.

[triggerthehorizon]

I tried the Android core and it works really well. I tried using the remastered theme from scummvm 2.5.0 and for some reason the mouse cursor appears all blocky. Also, I can not get the game Hopkins FBI to run, which was an issue on libretro ScummVM 2.10 as well. Any ideas on how to fix the mouse cursor and the Hopkins FBI game? I should also mention that I also copied all the engine data from Scummvm 2.5.0 to my Extras folder in retroarch

[DrUm78]

@triggerthehorizon yeah I had to remove this code at the end of those lines in "gui/themes/scummremastered/remastered_gfx.stx" to fix the mouse cursor on my repo.

From:

        <bitmap filename = 'logo.bmp' scalable_file = 'logo.svg' width = '287' height = '80'/>
        <bitmap filename = 'cursor.bmp' scalable_file = 'cursor.svg' width = '24' height = '24'/>
        <bitmap filename = 'cursor_small.bmp' scalable_file = 'cursor.svg' width = '16' height = '16'/>
        <bitmap filename = 'checkbox.bmp' scalable_file = 'checkbox.svg' width = '15' height = '15'/>
        <bitmap filename = 'checkbox_empty.bmp' scalable_file = 'checkbox_empty.svg' width = '15' height = '15'/>
        <bitmap filename = 'checkbox_disabled.bmp' scalable_file = 'checkbox_disabled.svg' width = '15' height = '15'/>
        <bitmap filename = 'radiobutton.bmp' scalable_file = 'radiobutton.svg'width = '15' height = '15'/>
        <bitmap filename = 'radiobutton_empty.bmp' scalable_file = 'radiobutton_empty.svg'width = '15' height = '15'/>
        <bitmap filename = 'logo_small.bmp' scalable_file = 'logo.svg' width = '144' height = '40'/>
        <bitmap filename = 'search.bmp' scalable_file = 'search.svg' width = '16' height = '16'/>
        <bitmap filename = 'eraser.bmp' scalable_file = 'eraser.svg' width = '16' height = '16'/>
        <bitmap filename = 'delbtn.bmp' scalable_file = 'delbtn.svg' width = '16' height = '16'/>
        <bitmap filename = 'list.bmp' scalable_file = 'list.svg' width = '16' height = '16'/>
        <bitmap filename = 'grid.bmp' scalable_file = 'grid.svg' width = '16' height = '16'/>
        <bitmap filename = 'stopbtn.bmp' scalable_file = 'stopbtn.svg' width = '32' height = '32'/>
        <bitmap filename = 'editbtn.bmp' scalable_file = 'editbtn.svg' width = '32' height = '32'/>
        <bitmap filename = 'switchbtn.bmp' scalable_file = 'switchbtn.svg' width = '32' height = '32'/>
        <bitmap filename = 'fastreplay.bmp' scalable_file = 'fastreplay.svg' width = '32' height = '32'/>
        <bitmap filename = 'stopbtn_small.bmp' scalable_file = 'stopbtn.svg' width = '16' height = '16'/>
        <bitmap filename = 'editbtn_small.bmp' scalable_file = 'editbtn.svg' width = '16' height = '16'/>
        <bitmap filename = 'switchbtn_small.bmp' scalable_file = 'switchbtn.svg' width = '16' height = '16'/>
        <bitmap filename = 'fastreplay_small.bmp' scalable_file = 'fastreplay.svg' width = '16' height = '16'/>

To:

        <bitmap filename = 'logo.bmp'/>
        <bitmap filename = 'cursor.bmp'/>
        <bitmap filename = 'cursor_small.bmp'/>
        <bitmap filename = 'checkbox.bmp'/>
        <bitmap filename = 'checkbox_empty.bmp'/>
        <bitmap filename = 'checkbox_disabled.bmp'/>
        <bitmap filename = 'radiobutton.bmp'/>
        <bitmap filename = 'radiobutton_empty.bmp'/>
        <bitmap filename = 'logo_small.bmp'/>
        <bitmap filename = 'search.bmp'/>
        <bitmap filename = 'eraser.bmp'/>
        <bitmap filename = 'delbtn.bmp'/>
        <bitmap filename = 'list.bmp'/>
        <bitmap filename = 'grid.bmp'/>
        <bitmap filename = 'stopbtn.bmp'/>
        <bitmap filename = 'editbtn.bmp'/>
        <bitmap filename = 'switchbtn.bmp'/>
        <bitmap filename = 'fastreplay.bmp'/>
        <bitmap filename = 'stopbtn_small.bmp'/>
        <bitmap filename = 'editbtn_small.bmp'/>
        <bitmap filename = 'switchbtn_small.bmp'/>
        <bitmap filename = 'fastreplay_small.bmp'/>

[DrUm78]

Hopkins FBI works for me (but very slowly as I'm on a low-end device), you need to extract the files from the ISO and not use the .cue/.bin version.

[triggerthehorizon]

Hopkins FBI works for me (but very slowly as I'm on a low-end device), you need to extract the files from the ISO and not use the .cue/.bin version.

Thanks DrUm78! Those edits fixed my mouse cursor :). What version are you using? I am using the Windows version. My files work fine in ScummVM standalone, it's only in the RA android core that it crashes on boot. Any idea what else could be wrong?

[DrUm78]

I use Windows CD-Rom version with the following file structure:

Hard to tell what your problem is without any log as it works for me.

[triggerthehorizon]

We have the same version. weird...

[DrUm78]

What file do you use to launch the game? Blank .scummvm? Because there is a bug with lower case file extension: https://github.com/libretro/scummvm/issues/189, I have to name it *.SCUMMVM (upper case) or it does not boot.

[triggerthehorizon]

It fails even if I load it from the scummvm launcher in retroarch

---

From: DrUm78 @.> Sent: Monday, November 15, 2021 4:59 PM To: libretro/scummvm @.> Cc: vmillar triggerthehorizon.com @.>; Mention @.> Subject: Re: [libretro/scummvm] Updated ScummVM core based on latest sources available if you want it (#179)

What file do you use to launch the game? Blank .scummvm? Because there is a bug with lower case file extension, I have to name it *.SCUMMVM (upper case) or it does not boot.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/libretro/scummvm/issues/179#issuecomment-969361100, or unsubscribehttps://github.com/notifications/unsubscribe-auth/APQ7RQP6STH5WNGIHV43WFTUMF7FXANCNFSM45NRXMIQ. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.