Image issues with Flycast core

[AltoRetrato]

Hi, John!

Your UnrealLibretro is amazing! Thank you so much for sharing this with us! 😄

I just tested it with a couple of cores, and it seems to work perfectly with some of them, e.g., dosbox_svn_libretro.

OTOH, the Flycast core has sound and controllers working fine (just like in Retroarch), but the image is distorted:

 

Expected output (captured with the same core on Retroarch):  

Other details:

• UnrealLibretro v0.2.1-alpha, UE 4.27.2
• Core: flycast_libretro
• ROM: Samba de Amigo for the Dreamcast, PAL version
• ROM: Samba de Amigo 2K for the Dreamcast, Japanese / NTSC
• Windows 10 (21H1)
• Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz 3.40 GHz
• NVIDIA GeForce GTX 1660

Any idea of what is causing the issue, and what would be the solution?

Thanks!

[AltoRetrato]

I added some debug messages in the middle of UnrealLibretro Delegates, hoping it might help understand the issue. It seems Flycast might be applying a "widescreen cheat" (see code at the bottom of this post).

Blueprint screenshot (for context)


Here's the slightly edited output log, after running for a few seconds: warning lines start with "**", debug messages with "##", repeated lines removed and marked with "[\<repetitions>x]".

Output Log (93 lines)

``` LogTemp: Repeating last play command: Selected Viewport LogPlayLevel: PlayLevel: No blueprints needed recompiling PIE: New page: PIE session: LibretroWorld (Jul 3, 2022, 8:16:03 AM) LogPlayLevel: Creating play world package: /Game/UnrealLibretro/UEDPIE_0_LibretroWorld LogPlayLevel: PIE: StaticDuplicateObject took: (0.002715s) LogAIModule: Creating AISystem for world LibretroWorld LogPlayLevel: PIE: World Init took: (0.001143s) LogPlayLevel: PIE: Created PIE world by copying editor world from /Game/UnrealLibretro/LibretroWorld.LibretroWorld to /Game/UnrealLibretro/UEDPIE_0_LibretroWorld.LibretroWorld (0.004020s) LogUObjectHash: Compacting FUObjectHashTables data took 4.31ms LogAudio: Display: Creating Audio Device: Id: 4, Scope: Unique, Realtime: True LogAudioMixer: Display: Audio Mixer Platform Settings: LogAudioMixer: Display: Sample Rate: 48000 LogAudioMixer: Display: Callback Buffer Frame Size Requested: 1024 LogAudioMixer: Display: Callback Buffer Frame Size To Use: 1024 LogAudioMixer: Display: Number of buffers to queue: 2 LogAudioMixer: Display: Max Channels (voices): 32 LogAudioMixer: Display: Number of Async Source Workers: 0 LogAudio: Display: AudioDevice MaxSources: 32 LogAudio: Display: Audio Spatialization Plugin: None (built-in). LogAudio: Display: Audio Reverb Plugin: None (built-in). LogAudio: Display: Audio Occlusion Plugin: None (built-in). LogAudioMixer: Display: Initializing audio mixer. LogAudioMixer: Display: 0: FrontLeft LogAudioMixer: Display: 1: FrontRight LogAudioMixer: Display: 2: FrontCenter LogAudioMixer: Display: 3: LowFrequency LogAudioMixer: Display: 4: SideLeft LogAudioMixer: Display: 5: SideRight LogAudioMixer: Display: Using Audio Device Speakers (Realtek High Definition Audio) LogAudioMixer: Display: Initializing Sound Submixes... LogAudioMixer: Display: Creating Master Submix 'MasterSubmixDefault' LogAudioMixer: Display: Creating Master Submix 'MasterReverbSubmixDefault' LogAudioMixer: Display: Creating Master Submix 'MasterEQSubmixDefault' LogAudioMixer: FMixerPlatformXAudio2::StartAudioStream() called LogAudioMixer: Display: Output buffers initialized: Frames=1024, Channels=6, Samples=6144 LogAudioMixer: Display: Starting AudioMixerPlatformInterface::RunInternal() LogAudioMixer: Display: FMixerPlatformXAudio2::SubmitBuffer() called for the first time LogInit: FAudioDevice initialized. LogAudio: Display: Audio Device (ID: 4) registered with world 'LibretroWorld'. LogLoad: Game class is 'LibretroGamemode_C' LogWorld: Bringing World /Game/UnrealLibretro/UEDPIE_0_LibretroWorld.LibretroWorld up for play (max tick rate 0) at 2022.07.03-11.16.03 LogWorld: Bringing up level for play took: 0.001124 LogOnline: OSS: Creating online subsystem instance for: :Context_5 PIE: Server logged in PIE: Play in editor total start time 0.116 seconds. ** Libretro: Warning: Unhandled env #52 ** Libretro: Warning: Unhandled env #69 Libretro: shell/libretro/libretro.cpp:308 N[BOOT]: retro_init ** Libretro: Warning: Unhandled env #28 ** Libretro: Warning: Unhandled env #65587 ** Libretro: Warning: Unhandled env #57 ** Libretro: Warning: Unhandled env #13 Libretro: core/hw/mem/_vmem.cpp:494 N[VMEM]: Info: nvmem is enabled, with addr space of size 512MB Libretro: core/hw/mem/_vmem.cpp:589 N[VMEM]: BASE 00007ff42dda0000 RAM(16 MB) 00007ff439da0000 VRAM64(8 MB) 00007ff431da0000 ARAM(2 MB) 00007ff44dda0000 Libretro: shell/libretro/libretro.cpp:1785 N[BOOT]: retro_load_game: E:/Unreal/LibretroVR/Plugins/MyROMs/sda_pal/Samba De Amigo v1.002 (2000)(Sega)(PAL)[!].gdi ** Libretro: Warning: Unhandled env #23 ** Libretro: Warning: Unhandled env #8388610 ** Libretro: Warning: Unhandled env #8388612 ** [69x] Libretro: Warning: Unhandled env #55 Libretro: File extension is: .gdi Libretro: core/hw/mem/_vmem.cpp:494 N[VMEM]: Info: nvmem is enabled, with addr space of size 512MB Libretro: core/hw/mem/_vmem.cpp:589 N[VMEM]: BASE 00007ff42dda0000 RAM(16 MB) 00007ff439da0000 VRAM64(8 MB) 00007ff431da0000 ARAM(2 MB) 00007ff44dda0000 Libretro: core/emulator.cpp:55 N[BOOT]: Game ID is [MK-5109250] Libretro: core/emulator.cpp:479 N[BOOT]: Did not load BIOS, using reios ** Libretro: Warning: Unhandled env #1 ** [69x] Libretro: Warning: Unhandled env #55 Libretro: shell/libretro/libretro.cpp:2120 N[RENDERER]: retro_get_system_av_info: Res=480 ## LogBlueprintUserMessages: [LibretroTVActor_2] OnCoreFrameBufferResize: Scale Fill: 0, 0, Rotation Hz: 0 Libretro: GL_SHADING_LANGUAGE_VERSION: 1.50 NVIDIA via Cg compiler Libretro: GL_VERSION: 3.2.0 NVIDIA 516.59 Libretro: core/rend/gles/gles.cpp:544 N[RENDERER]: Open GL version 3.2 ** Libretro: Warning: Unhandled env #65585 ## LogBlueprintUserMessages: [LibretroTVActor_2] OnCoreIsReady: Address: Wrap,Wrap, Auto Generate Mips: false, Mips Address: Clamp,Clamp, Mips Sample Filter: Default (from Texture Group), Render Target Format: RTF RGBA16f, Shared: true, Size: 853,853 ## LogBlueprintUserMessages: [LibretroTVActor_2] OnCoreFrameBufferResize: Scale Fill: 0.75, -0.563, Rotation Hz: 0 Libretro: core/reios/reios.cpp:622 N[REIOS]: ----------------- Libretro: core/reios/reios.cpp:623 N[REIOS]: REIOS: Booting up Libretro: core/reios/reios.cpp:624 N[REIOS]: ----------------- ** [277x] Libretro: Warning: Unhandled env #65585 LogSlate: Updating window title bar state: overlay mode, drag disabled, window buttons hidden, title bar hidden LogWorld: BeginTearingDown for /Game/UnrealLibretro/UEDPIE_0_LibretroWorld LogWorld: UWorld::CleanupWorld for LibretroWorld, bSessionEnded=true, bCleanupResources=true LogSlate: InvalidateAllWidgets triggered. All widgets were invalidated ** Libretro: Warning: Unhandled env #65585 LogPlayLevel: Display: Shutting down PIE online subsystems LogSlate: InvalidateAllWidgets triggered. All widgets were invalidated ** Libretro: Warning: Unhandled env #65585 LogAudio: Display: Audio Device unregistered from world 'None'. LogAudioMixer: FMixerPlatformXAudio2::StopAudioStream() called ** Libretro: Warning: Unhandled env #65585 LogAudioMixer: FMixerPlatformXAudio2::StopAudioStream() called LogSlate: Updating window title bar state: overlay mode, drag disabled, window buttons hidden, title bar hidden LogUObjectHash: Compacting FUObjectHashTables data took 3.13ms LogPlayLevel: Display: Destroying online subsystem :Context_5 ```

From the log, I've noticed that the Libretro Framebuffer size is 853 × 853, which seems a bit odd. I saved the Emulator Texture to disk, and while the resolution is 853 × 853, the contents are 640 × 480.

Emulator Texture


I trimmed the image above to 640 × 480 and saved it (as a "raw" file), then loaded it as if it had 853 pixels of width - and it appeared as "normal" (only vertically flipped).

"Unwrapped" Emulator Texture


So it seems 853 pixels per line are being written in the 853 × 853 buffer, but they are wrapping at 640 pixels.

In the log it says the RenderResolution is 480: Libretro: shell/libretro/libretro.cpp:2120 N[RENDERER]: retro_get_system_av_info: Res=480

In flycast's retro_get_system_av_info there is a mention to a widescreen cheat, but the same core and same ROM run in 4:3 in Retroarch.

But in setFramebufferSize:

static void setFramebufferSize()
{
    framebufferHeight = config::RenderResolution;
    if (config::Widescreen)
        framebufferWidth = config::RenderResolution * 16.f / 9.f;
    else if (!rotate_screen)
        framebufferWidth = config::RenderResolution * 4.f * config::ScreenStretching / 3.f / 100.f;
    else
        framebufferWidth = config::RenderResolution * 4.f / 3.f;
    maxFramebufferHeight = std::max(maxFramebufferHeight, framebufferHeight);
    maxFramebufferWidth = std::max(maxFramebufferWidth, framebufferWidth);
}

the line framebufferWidth = config::RenderResolution * 16.f / 9.f; means that if config::Widescreen is true, framebufferWidth is 480 * 16 / 9, i.e., 853.33.

And right below, in setGameGeometry, it seems geometry.base_width and geometry.base_height are hardcoded:

static void setGameGeometry(retro_game_geometry& geometry)
{
    geometry.aspect_ratio = (float)framebufferWidth / framebufferHeight;
    if (rotate_screen)
        geometry.aspect_ratio = 1 / geometry.aspect_ratio;
    geometry.max_width = std::max(framebufferHeight * 16 / 9, framebufferWidth);
    geometry.max_height = geometry.max_width;
    // Avoid gigantic window size at startup
    geometry.base_width = 640;
    geometry.base_height = 480;
}

I'm not sure, but does it mean that a possible fix would be using geometry.max_width and geometry.max_height in ULibretroCoreInstance::Launch(), instead of geometry.base_width and geometry.base_height?

[macdguy]

Following @AltoRetrato findings, I was able to find a fix.

The issue I found was definitely with the sizing of the render target. However, replacing geometry.base_width and geometry.base_height in ULibretroCoreInstance::Launch() is only part of the fix.

Replacing every instance of max_width and max_height in sdlarch.cpp with base_width and base_height resolved the issue. I'm unsure if this affects games that adjust the aspect ratio or rotate the screen.

This change seemed to have no impact on other cores from my testing, and I tested with Snes9x, Mupen 64, Nestopia.

Also, to fix Flycast being flipped, you could add a material parameter to the material then flip it just for Flycast, or add some code to toggle invert_y's value in LibretroCoreInstance.h

Edit: Upon further testing, this does break compatibility with some cores.

[N7Alpha]

Thanks for the detailed bug report @AltoRetrato and attempting to fix it @macdguy. I appreciate it. I think I found the issue. The first part was the pitch I was using when copying the framebuffer was wrong that explained the weird interlaced shifted look it had.

The other issue was I misinterpreted where the frame would be rendered into the framebuffer when retro_hw_render_callback::bottom_left_origin was true. I thought it would be rendered in the pink box below. This wasn't too hard to solve I just had to shift the uv coordinates however I moved the logic to the blueprints which aren't under source control yet. So you'll have to run the setup.sh script again or just copy over the blueprints from a build artifact or the unversioned branch to see the changes there.

I think this solved it. Just close the issue if it works

[AltoRetrato]

I was having issues building the plugin at first, but I managed to build it with the Unreal AutomationTool, via RunUAT.bat!

The batch is in <UE_4.27_install_path>\Engine\Build\BatchFiles . From there I run RunUAT.bat BuildPlugin -plugin="E:\Unreal\LibretroVR\Plugins\UnrealLibretro.uplugin" -package="E:\Unreal\LibretroVR\Plugins\Output" -TargetPlatforms=Win64 -VS2019 (without the -VS2019 parameter, it complained that I needed Visual Studio 2017, but with -VS2019 it worked with VS 2022 just fine).

After that, everything was working properly! Thank you! 😄