Preparations for latest version of Amiberry

[midwan]

As we're getting close to a new stable release of Amiberry, I thought we should collaborate on bringing that to RetroPie as well!

Things that have changes since the older version (v2.1):

• New lib dependency, for CD32 FMV support
• new file dependencies under data/ folder (included in the amiberry repo)
• SDL1 and SDL2 versions are supported as targets, in the same Makefile. SDL1 uses Dispmanx as before, the SDL2 versions have two flavors: One with Dispmanx (for best performance) and one with pure SDL2 (using either "rpi" or "kmsdrm" drivers). Ideally we'd love to move to pure SDL2 with KMS, but the performance is still worse with that combination, so we're keeping the Dispmanx version around.
• Config file cleanup and updates. Older config files should work, but you'd be left with some junk there. Ideally you'd want to use clean, new config files.

The emulator updates themselves include tons of improvements, bugfixes, optimizations etc. We're now based on WinUAE 3.6.0, there's support for Quickstart configs, full controller remapping, Retroarch config file support, full controller navigation in the GUI, and many, many more.

I'd love to assist in any way needed, to ensure a smooth transition to the latest version.

Note: The latest version is currently still sitting on the "dev" branch, but will soon be moved to "master". We have setup a wiki regarding several documentation items (e.g. controller mapping, retroarch support, how to compile SDL2 from source with KMS support), in case it helps.

Feel free to let me know if you need anything special. Beta binaries can be provided if you need them.

[psyke83]

I can set up a PR to update the current scriptmodule so that it builds properly against the current "dev" branch (and switch it to master when needed), but for simplicity sake, it might be best to focus on a providing a single version. A KMS build is not realistic for RetroPie right now, as it breaks a lot of functionality compared to the vendor driver (hw-accelerated decoding, tvservice, etc.,) and software such as Kodi aren't available in Raspbian's repositories compiled against the Mesa headers.

We're trying to move away from SDL1 software when possible, so either the dispmanx or pure SDL2 version (rpiX-sdl2?) might be the best choice?

[midwan]

@psyke83 Does it also break with the Fake KMS driver?

You can go for SDL1 + Dispmanx (which is what the previous version used also) or for SDL2 + Dispmanx. Or you could also choose SDL2 with the "rpi" driver back-end. However, I noticed that the best performance currently comes from one of the Dispmanx combinations above (either SDL1 or SDL2).

If you want SDL2, the extra dependency is to have SDL2 with either the "rpi" or "kms" driver enabled, though to use the KMS driver you'll have to use the fkms or kms drivers in the system (which I understand will break things). SDL2 graphics will only be used for the GUI, then Dispmanx is used for actual emulation.

Maybe it's best to test it out and decide which one works best?

[psyke83]

The fake KMS does allow some partial support for tvservice etc., but there are too many issues to make it viable for widespread use, not to mention the fact that all software still needs to be recompiled against the Mesa headers instead of the vendor (/opt/vc) ones, and repository-sourced software such as VLC or Kodi don't provide compatible packages right now.

I've been reading on the Raspberry Pi forums that they plan to move to the 4.14 kernel some time in the near future. This ought to be good news, as most development we've been seeing related to f/kms has been through selective backports from newer releases to to the 4.9 kernel. However, even when they switch to the new kernel, the majority of the issues probably won't be addressed with the new driver for some time. We'll most likely be using the old graphics libraries for the foreseeable future.

I'll try some builds against jessie and stretch against your dev branch, but I'm leaning towards testing SDL2. Pending successful testing, I'll be sure to tag you in the PR to get feedback or suggestions. Thanks!

[midwan]

@psyke83 How is this coming along? Need any help with anything? Since we seem to have all the critical bugs out of the way now, we're aiming towards a new stable release (merging dev into master in the process). I'm thinking of setting a deadline around the end of this week. Would that be OK with you?

From our tests, it seems that the fastest option for the Pi is currently SDL2 (compiled with --disable-video-rpi --enable-video-kmsdrm) with the Dispmanx back-end. I'm using fkmshere, to be able to run the kmsdrm option. The relevant Makefile target is PLATFORM=rpi3-sdl2-dispmanx. If you prefer the Legacy driver for better compatibility with other systems, then perhaps you should go for the rpi driver instead and skip the --enable-video-kmsdrm altogether. Don't try to enable both of them, as from my experience they don't work well together (got a black screen when I did that).

Using pure SDL2 with KMSDRM is still much slower than hitting Dispmanx directly, and we cannot use multi-threaded rendering unless we're running under x11. That is not the case with other platforms (e.g. tested on the Tinker board and multi-threaded rendering works with KMSDRM) so it must be related to how the backend is implemented on the Pi... :-(

[psyke83]

@midwan,

I haven't yet had a chance to compare performance differences between builds, but some of your comments have piqued my interest.

RetroPie uses a custom version of SDL which is currently built from the rpi-2.0.7 branch. As you can see from the debian rules file, it's compiled only with the RPI driver, and considering my comments re: compatibility issues with software such as Kodi, kms or fkms support is currently out of the question.

Putting that aside that fact, I am curious about what you said re: kmsdrm. Some patches that I submitted upstream should allow the rpi and kmsdrm drivers to coexist when both drivers are built into the same library. When I was testing the kmsdrm driver at the time of submitting those patches, the appropriate driver would be used depending on the active kernel overlay; so f/kms mode would use the kmsdrm SDL driver automatically, and the default or legacy mode would use the rpi driver. The actual program needed to be compiled against the appropriate headers, however.

When targeting KMS, are you compiling Amiberry against the Broadcom headers in /opt/vc/include? Surely that can't be the right approach, as you're supposed to target the generic Mesa headers in /usr/include/EGL and so on. Am I missing something?

[psyke83]

Ah - I recall now that my coexistence patches were submitted after the 2.0.7 release window, so if you're not using the latest SDL development snapshot, it would explain your issue. I would suggest that you apply the same patches against the 2.0.7 SDL source (or perhaps fork the rpi-2.0.7 branch which contains these patches, but you'd need to modify the debian rules so it builds both drivers).

[psyke83]

I just compiled a new build, and both sdl2-dev and sdl2-dispmanx-dev has an issue in which the GUI is only ~50% the size of the full screen, positioned at the bottom-left of the screen, but the cursor is restricted to the same size of the window starting from the top-left of the screen, meaning that most elements can't be selected.

bisected to: https://github.com/midwan/amiberry/commit/cd3998cc4f53106000130ed491fc3b86e395cd0d Removing this part of the patch seems to resolve the GUI issue: https://github.com/midwan/amiberry/commit/cd3998cc4f53106000130ed491fc3b86e395cd0d#diff-d24ab0dcd90938a4c3f7f659b95af05aR299

I don't have time right now to test for issues beyond the GUI, but will look at it soon.

[midwan]

@psyke83 Thanks for the comments about SDL2. I think when I found this problem I was still using 2.0.7, so it would explain the situation. Amiberry is only built with the /opt/vc stuff if you target Dispmanx, otherwise it won't reference them.

I've since switched to the current dev version of SDL, directly from their Mercurial repo. I'll have to re-compile it to test this again.

I didn't notice the problem you mentioned, but I'll run another round of tests using different environments (e.g. RPI driver, KMSDRM driver, console startup, X11 startup) to see if I can recreate it.

[midwan]

@psyke83 I tested the build using RetroPie's provided SDL2 libs. I found that there is a problem using Dispmanx directly, if SDL2 is already using it with the rpi backend (which RetroPie's libs do). You'll get a black screen after the GUI in that case, probably because we're trying to initialize Dispmanx and use it directly while SDL2 has already grabbed hold of it...

On the other hand, the rpi3-sdl2 build does work. Speed will not be as good (because we also use multi-threaded rendering in Dispmanx, though there might be a way to make that work with the rpi backend as well) but it will work.

Remember to check the Fullscreen option in the Display panel, otherwise the screen will be positioned on the top-left corner. This is again an rpi backend behavior (KMSDRMwill use FULL_SCREEN_DESKTOPby default). Enabling Full screen will center the image correctly however (we switch from WINDOWEDto FULL_SCREENin SDL2 using that checkbox).

I didn't test the mouse yet, because in my current test system I didn't have a mouse connected. I had a rather small screen also (800x480, official RPI touchscreen) so I didn't notice any problem with the GUI as you mentioned, but that may be because of my resolution (the GUI is designed for exactly that size as a minimum). This may be an issue with the rpi backend and the FULLSCREENmode I mentioned above actually - since it starts up in WINDOWEDmode by default.

[psyke83]

The default configuration has fullscreen set to false, but enabling it, saving to the default uaeconfig and restarting doesn't resolve the issue.

I couldn't test 800x600 on my TV, but on 720x480, the entire screen is filled and the mouse is not bounded differently to the visible screen (good!), but the rightmost part of the GUI is cut off.

As I already mentioned, removing this entire block of code in src/osdep/gui/main_window.cpp resolves the issue with my TV's default resolution (1366x768):

                if (sdlWindow)
                {
                        if ((SDL_GetWindowFlags(sdlWindow) & SDL_WINDOW_MAXIMIZED) == 0)
                                SDL_SetWindowSize(sdlWindow, GUI_WIDTH, GUI_HEIGHT);
                }

Removing this will also resolve the issue with the rightmost part of the GUI being cropped when running in 720x480. I recommend that you connect a larger screen to test this issue, as the GUI is completely unusable at 720p+ resolutions due to the incorrect mouse bounding.

Thanks.

As for dispmanx, I'll be happy to amend the PR to use the dispmanx build if that's what you'd prefer, but the conflict with the SDL videodriver would need to be resolved - we're not going to change our SDL2 build configuration just for amiberry.

Is it possible to workaround the conflict by specifying a different driver via the SDL_VIDEODRIVER environment variable, perhaps?

[psyke83]

Another thought: for a build running on a Pi via RetroPie, the concept of windowed mode doesn't make sense, as we don't run in an X11 context with a window manager.

Regarding the above code which is causing problem, it might make sense to somehow detect configurations that run in KMS or dispmanx context and disable the ability to set windowed mode (including fullscreen toggle). Existing calls to SDL_SetWindowFullscreen may also need to use the SDL_WINDOW_FULLSCREEN_DESKTOP flag instead of SDL_WINDOW_FULLSCREEN, but maybe I'm wrong... I'm no expert with SDL2 application development.

[midwan]

@psyke83 I will make a change to detect the backend we're running under, and only enable windowed mode if running under "x11". I believe that will fix the issue above.

However, we still won't be able to use the Dispmanx version because SDL2 will have already initialized it. We'll have to see how that affects things regarding performance.

[midwan]

@psyke83 I've modified a few things to hopefully fix these problems... There's now a check for the Video Driver used. If it's "x11", and only then, the Windowed/Fullscreen options will take effect. Otherwise we go for FULLSCREEN_DESKTOP.

This has been applied to the GUI and the emulation screen as well. I will even disable the Fullscreen checkbox in the GUI in those scenarios to avoid the confusion.

I tested it on Raspbian using the Legacy driver and SDL2 compiled with the rpi driver. It seems to work correctly now, both in the GUI and in the emulation screen.

Regarding the Dispmanx version: it can only work if we are using a different backend than rpi in SDL2, as far as I could see from my tests. I'm not sure there's anything we can do about that, as we're trying to open and use Dispmanx (in a separate thread also) directly from Amiberry to maximize performance - but SDL2 with the rpi backend also initializes it and uses it, and since that comes first (for the GUI) the emulation screen that is opened afterwards is black. One possible workaround would be to have a version that uses Dispmanx exclusively for all drawing, even for the GUI, in SDL2. Then we'd be basically using SDL2 for everything else except video. But as that would be RPI-specific, and considering that Dispmanx is considered deprecated, I'm not sure if it's worth the effort.

I guess the Dispmanx version rules out your environment for now, unless you select the SDL1 version (which is what the older version was doing also). If Retropie uses the "fkms" driver in the future, and SDL2 is compiled with --enable-video-kmsdrm, then it can be used. Otherwise, if we want to use the SDL2 version with the rpi backend, the only target that works correctly would be rpi3-sdl2 (or rpi2-sdl2 etc).

I'll commit these changes as soon as I've finished testing the other backends and made sure there are no problems there as well.

[midwan]

@psyke83 OK, the changes are committed in my repo.

Note: The fastest possible option on RetroPie still is SDL1 + Dispmanx (target: rpi3), since we can't use SDL2 + Dispmanx due to the rpi backend. If something changes in this situation, I'll update you accordingly. :)

[midwan]

@psyke83 Just to clarify a bit further, depending on the video driver each distro is using, we can use the relevant target for best performance:

• Legacy driver: SDL1 + Dispmanx
• FKMS driver: SDL2 (compiled with --disable-video-rpi --enable-video-kmsdrm) + Dispmanx
• KMS driver, or other platforms: SDL2 (without Dispmanx)

I hope the above helps!

[psyke83]

Thanks! I've been comparing performance, and unfortunately it does seem that the plain SDL2 build is a fair bit slower (I'm testing with Alien Breed 3D II).

I took a look at your source to see the multithreaded rendering you mentioned, and it looks like the USE_RENDER_THREAD encapsulated code is not going to work; from: https://wiki.libsdl.org/CategoryThread

NOTE: You should not expect to be able to create a window, render, or receive events on any thread other than the main one. For platform-specific exceptions or complicated options ask on the forums/mailing list.

I tried moving more of what's done in the separate function back into the main thread, but it seems that not even one of the operations complete properly even though the thread is executing properly. The question is - does the multithreading code actually work on tinker, and if so, why?

Have you tried experimenting with the dispmanx scaling? We use it for mupen64plus on RetroPie, but I didn't have much luck on amiberry. Maybe that's because amiberry doesn't set up a lower screen resolution compared to the native resolution?

I'm still confused as to why your FKMS case works. FKMS is an emulated KMS session in a dispmanx context (to help support vendor features like tvservice until VC4 is feature complete), but I thought that userland applications were not supposed to be compiled with dispmanx support for either type of KMS mode?

It seems that SDL1 + Dispmanx performs best on Pi. If we stick to that for Pi instead of SDL2, would there be any limitations imposed, such as the new joystick support?

[midwan]

@psyke83 Thanks for the feedback!

The render thread code does work on the Tinkerboard, and it also works on the Pi if using the x11 backend, but only then. Therefore I have to assume that's it's because of differences in the different backends and their implementations. If it worked on the KMSDRM backend on the Pi as well, we'd have a much better solution, but sadly it does not.

I haven't tested the dispmanx scaling you mentioned, but we already do scaling using dispmanx in the code as it is (scaling whatever Amiga resolution is opened to fullscreen, optionally keeping the correct aspect ratio). I'll see if there's something there that makes things better :)

Fkms seems to allow applications that use dispmanx directly to continue working (like Amiberry does), but I'm not sure why/how. I reached that by experimenting with the different backends and video driver options :)

It's ok to stick to SDL1 + Dispmanx for now, the code is identical between versions and almost all the features are there as well. Going forward however, I was considering removing support for SDL1 to focus on SDL2 exclusively. Of course, you have to consider the practical implications of such a decision, so if the performance suffers because of this, we will wait.

For now, the approach of FKMS and SDL2 + Dispmanx seem to be identical or better in performance than the Legacy and SDL1 + Dispmanx option, so it seems we have a way forward. Of course, that means that for a full distro like RetroPie, everything else needs to work under FKMS as well, which is another big task by itself...

[psyke83]

I notice that you've merged your work onto the master branch and dropped the -dev suffix. Considering this, and our discussion regarding the appropriate default build for Pi, I've updated the PR:

• The scriptmodule should allow building for tinker and odroid-xu (the latter platform renamed to xu4-sdl2 to match your Makefile), but I don't own either of these boards to test.
• For all targets except Pi, it builds the -sdl2 version.
• For Pi, it builds the -sdl1 version with the default legacy driver, or -sdl2 if kms is detected*.

*Just to re-iterate, KMS is not a supported configuration on Pi yet. I know that you prefer the -sdl2-dispmanxoption for this configuration, but we can revisit that if/when we add official KMS support to the script. If that happens, our scripts would need adjusting to support detection of fake vs real kms, because it currently detects both cases simply as kms.

[midwan]

@psyke83 I was just about to update you on that. :)

Thanks for making the changes.

For the Pi, building the sdl1 (+dispmanx) version is OK. The -sdl2-dispmanx should be used if only with the fkms driver, and the -sdl2 version with anything else (kms). Since you don't use fkms/kms yet however, it's not a problem. We can revisit this when RetroPie moves to a newer version that supports it.

[midwan]

For the record, I've also uploaded binaries with the v2.11 release. The "Jessie" binaries for SDL1 have been compiled/tested on the current RetroPie system.

[midwan]

@psyke83 Not sure if you have any installations on RPI1/Zero, but I've found a problem when compiling Amiberry for that arch. It will be fixed with an update, but for now just keep that in mind...

[psyke83]

@midwan - feel free to reopen this issue if you encounter any problems with RetroPie integration, or there's any updates regarding SDL2 dispmanx being more compatible with the rpi SDL2 driver, or the RPI1 build needing fixing via the script side.

When I have free time I'll investigate the suggestions regarding the conf path (which I'm not convinced is a good idea, if .uae files manually placed in roms/amiga can be launched OK). Thanks.

[midwan]

@psyke83 Thanks for the help with this! Is it already available for RetroPie users? I have several people asking/waiting for that... Maybe I can tell them the good news? :)

[psyke83]

If they update their RetroPie-Setup script, then they can install the build via source right now.

The "install from binary" will not provide the latest version until @joolswills manually triggers and uploads the builds to the RetroPie server. Unfortunately, I can't give you the ETA for that as Jools is very busy, and apart from his actual work, we're currently in the process of ironing out bugs for general stretch support. Besides, it'd probably be wise to wait for the RPI1 build to be sorted so that all the binaries can be generated (as RPI1/Zero are targets we support).

[midwan]

OK, good to know, thanks!

[midwan]

Update: I'm pushing v2.14 out today, the sources are already merged and binaries are compiled at the moment.

PS: Do you want me to keep updating you on new releases, or do you monitor those yourself?

[chewabledrapery]

Hi,

I've updated Amiberry on my retropie (rpi3) to the latest version and now no games load I had originally followed guides and had it all working with WHDLOAD games.

Now when i launch anything it either bombs out or I just get stuck on a red flashing screen, booting to kick roms and workbench works just no WHDLOAD games.

Am i missing something?

[midwan]

@chewabledrapery This has been mentioned in the Facebook groups a few times, it's related to old config files about 99% of the time. UAE Config Maker was updated together with Amiberry's release to reflect those changes, so the config files need to be updated accordingly with it.