Platforms Show as Sub-Menus of themselves if System Path is Not Available

[Enverex]

This one is a little weird so bare with me as I try and explain this.

Setup: All assets and EmulationStation itself are on the system disk, all the actual games (and thus the system paths) are on a second disk which is mounted after boot (because it takes over 30 seconds to mount).

Issue: If the system paths are not yet available when EmulationStation is opened, ES seems to put the systems inside a subfolder of themselves in the interface. Normally, you have the systems, you select the system and it will take you to detail view where you can see all the games. If this issue has been triggered, when going from the systems list into a system will still take you to the detail view, but there will only be a single item listed matching the name of the system you're already looking at. If you then go in to that, it'll take you into the game list as you would have originally expected.

This is rather weird and only happens if the "path" that a system points to isn't available when ES is opened. Reopening ES after the path has become available (e.g. as the disk is now mounted) results in normal behaviour again.

I can provide pictures/video of this if the description doesn't make sense.

[cmitu]

What version of ES are you using/which OS ? Can you provide a log file after a start-up where this problem occurs ?

[Enverex]

Version is GIT master, compiled from source on that machine running on Linux (Arch). Shows as:

Version 2.8.0rp-dev, built Nov 27 2018 - 11:27:03

It's launched with the command:

emulationstation --gamelist-only --no-splash --vsync on

No errors are produced on the command line. Log file when it doesn't load properly shows:

lvl2:   EmulationStation - v2.8.0rp-dev, built Nov 27 2018 - 11:27:03
lvl2:   Parsing XML file "/home/arcade/.emulationstation/resources/mamenames.xml"...
lvl2:   Creating surface...
lvl2:   Created window successfully.
lvl2:   Added known joystick Microsoft X-Box 360 pad (instance ID: 0, device index: 0)
lvl2:   Added known joystick Microsoft X-Box 360 pad (instance ID: 1, device index: 1)
lvl2:   Added known joystick Microsoft X-Box 360 pad (instance ID: 2, device index: 2)
lvl2:   Added known joystick Microsoft X-Box 360 pad (instance ID: 3, device index: 3)
lvl2:   Checking available OpenGL extensions...
lvl2:    ARB_texture_non_power_of_two: ok
lvl2:   Loading system config file /home/arcade/.emulationstation/es_systems.cfg...
lvl1:     Unknown platform for system "amstradgx4000" (platform "amstradgx4000" from list "amstradgx4000")
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/amstradgx4000/gamelist.xml"...
lvl1:     Unknown platform for system "mame" (platform "mame" from list "mame")
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/mame/gamelist.xml"...
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/atari2600/gamelist.xml"...
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/atari5200/gamelist.xml"...
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/atari7800/gamelist.xml"...
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/atarijaguar/gamelist.xml"...
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/atarilynx/gamelist.xml"...
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/wonderswan/gamelist.xml"...
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/wonderswancolor/gamelist.xml"...
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/colecovision/gamelist.xml"...
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/c64/gamelist.xml"...
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/amiga/gamelist.xml"...
lvl1:     Unknown platform for system "amigacd32" (platform "amigacd32" from list "amigacd32")
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/amigacd32/gamelist.xml"...
lvl1:     Unknown platform for system "daphne" (platform "daphne" from list "daphne")
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/daphne/gamelist.xml"...
lvl1:     Unknown platform for system "futurepinball" (platform "futurepinball" from list "futurepinball")
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/futurepinball/gamelist.xml"...
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/vectrex/gamelist.xml"...
lvl1:     Unknown platform for system "pico8" (platform "pico8" from list "pico8")
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/pico8/gamelist.xml"...
lvl1:     Unknown platform for system "odyssey2" (platform "odyssey2" from list "odyssey2")
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/odyssey2/gamelist.xml"...
lvl1:     Unknown platform for system "dos" (platform "dos" from list "dos")
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/dos/gamelist.xml"...
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/msx/gamelist.xml"...
lvl1:     Unknown platform for system "msx2" (platform "msx2" from list "msx2")
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/msx2/gamelist.xml"...
lvl1:     Unknown platform for system "pcfx" (platform "pcfx" from list "pcfx")
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/pcfx/gamelist.xml"...
lvl1:   System "pcfx" has no games! Ignoring it.
lvl1:     Unknown platform for system "turbografx16" (platform "turbografx16" from list "turbografx16")
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/turbografx16/gamelist.xml"...
lvl1:     Unknown platform for system "turbografxcd" (platform "turbografxcd" from list "turbografxcd")
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/turbografxcd/gamelist.xml"...
lvl1:     Unknown platform for system "3ds" (platform "3ds" from list "3ds")
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/3ds/gamelist.xml"...
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/nds/gamelist.xml"...
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/gb/gamelist.xml"...
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/gba/gamelist.xml"...
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/gbc/gamelist.xml"...
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/gc/gamelist.xml"...
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/n64/gamelist.xml"...
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/nes/gamelist.xml"...
lvl1:     Unknown platform for system "nsview" (platform "nsview" from list "nsview")
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/nsview/gamelist.xml"...
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/snes/gamelist.xml"...
lvl1:     Unknown platform for system "switch" (platform "switch" from list "switch")
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/switch/gamelist.xml"...
lvl1:     Warning from theme "/home/arcade/.emulationstation/themes/romscan/switch/theme.xml"
    could not find file "./art.jpg" (which resolved to "/home/arcade/.emulationstation/themes/romscan/switch/art.jpg") 
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/wii/gamelist.xml"...
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/wiiu/gamelist.xml"...
lvl1:     Unknown platform for system "wiiware" (platform "wiiware" from list "wiiware")
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/wiiware/gamelist.xml"...
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/3do/gamelist.xml"...
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/pc/gamelist.xml"...
lvl1:     Unknown platform for system "cdi" (platform "cdi" from list "cdi")
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/cdi/gamelist.xml"...
lvl1:     Warning from theme "/home/arcade/.emulationstation/themes/romscan/cdi/theme.xml"
    could not find file "./logo.svg" (which resolved to "/home/arcade/.emulationstation/themes/romscan/cdi/logo.svg") 
lvl1:     Warning from theme "/home/arcade/.emulationstation/themes/romscan/cdi/theme.xml"
    could not find file "./logo.svg" (which resolved to "/home/arcade/.emulationstation/themes/romscan/cdi/logo.svg") 
lvl1:     Warning from theme "/home/arcade/.emulationstation/themes/romscan/cdi/theme.xml"
    could not find file "./logo.svg" (which resolved to "/home/arcade/.emulationstation/themes/romscan/cdi/logo.svg") 
lvl1:     Unknown platform for system "videopacplus" (platform "videopacplus" from list "videopacplus")
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/videopacplus/gamelist.xml"...
lvl1:     Unknown platform for system "scummvm" (platform "scummvm" from list "scummvm")
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/scummvm/gamelist.xml"...
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/sega32x/gamelist.xml"...
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/dreamcast/gamelist.xml"...
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/gamegear/gamelist.xml"...
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/mastersystem/gamelist.xml"...
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/megadrive/gamelist.xml"...
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/segacd/gamelist.xml"...
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/saturn/gamelist.xml"...
lvl1:     Unknown platform for system "x68000" (platform "x68000" from list "x68000")
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/x68000/gamelist.xml"...
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/ngpc/gamelist.xml"...
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/psx/gamelist.xml"...
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/ps2/gamelist.xml"...
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/ps3/gamelist.xml"...
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/psp/gamelist.xml"...
lvl1:     Unknown platform for system "system" (platform "system" from list "system")
lvl2:   Parsing XML file "/home/arcade/.emulationstation/gamelists/system/gamelist.xml"...

Running again with the disk mounted (wherein the systems/platforms are shown and work properly) creates an identical log (I diff'd the two just to make sure). So it looks like whatever causes this weird issue doesn't actually get logged.

[cmitu]

Does it happen if you remove the --gamelist-only argument ?

[Enverex]

Well that obviously wouldn't work because it would then try and scan for content which A) doesn't exist because the paths for the systems point to a location that isn't mounted yet (which is the trigger for what happens in the first place) and B) would take forever on a library this size.

It looks like --gamelist-only still has a hangover from the old code somewhere and it's still checking the paths when starting, rather than just absolutely trusting the gamelist.xml files to be correct. There was a bug long-ago where --gamelist-only didn't work at all, that was fixed (before being forked here) but it looks like there may be something leftover.

[pjft]

Well, I can suggest why it is having these results, but the root cause would need a bit more investigation.

Ultimately, it is related to whatever is causing the platforms to be “unknown”. I suspect it might certainly be related to the fact that the paths aren’t mounted when es_systems.cfg is parsed - I don’t know that code by heart but it wouldn’t shock me if a missing folder would cause the platform not to be initialized as such. The fact that the systems then show up as a folder inside a folder is probably a side-effect of those sub-systems being potentially treated as collections instead. I suspect that if you launch those games they won’t launch as it doesn’t know what emulator to call, would that be a correct assessment?

Certainly sharing screenshots for a single system can help, just to confirm what we’re assuming here, but as far as a fix for it goes I’m not sure. We could consider relieving the requirement (if present) for a path to exist in order to initialize a platform, but I’m not fully sure of the consequences for that. Assuming there is one such test/assertion and that the failure of accessing the path results in the platform not being initialized.

EDIT: from reading the code, there doesn’t seem to be one such assertion, but somewhere it seems that when it has errors it’s reading a completely different es_systems.cfg . Could you share the following:

• /home/arcade/.emulationstation/es_systems.cfg from a “good” ES launch;
• /home/arcade/.emulationstation/es_systems.cfg from a “bad” ES launch (unplug the drive altogether);
• The results of running ‘df’ in your shell;
• A copy of /etc/emulationstation/es_systems.cfg (I think that’s the backup path)

Thanks!

[Enverex]

Certainly sharing screenshots for a single system can help, just to confirm what we’re assuming here

So this is what I expect to see (and do if the disk is mounted):

But if the disk isn't mounted, I get this instead:

If I then select the "game" of "PC - Windows" I get the normal game list as per the first image.

We could consider relieving the requirement (if present) for a path to exist in order to initialize a platform, but I’m not fully sure of the consequences for that.

That's not really what I was aiming for here. When using "gamelist-only" mode it shouldn't be checking anything anywhere, the whole point of using an existing gamelist is that it trusts it to be correct, so the systems should show up in the frontend regardless. Obviously launching anything wouldn't work until the disk has finished mounting, but by the time someone has navigated to the system and game they want, the disk should have finished mounting and it'll work. Basically being able to load ES without worrying about whether the content is available or not yet.

/home/arcade/.emulationstation/es_systems.cfg from a “good” ES launch; /home/arcade/.emulationstation/es_systems.cfg from a “bad” ES launch (unplug the drive altogether);

This file is static and doesn't change, so it's the same in both scenarios (I tested, just in case). Attached for reference. The "IGNORE" bit is because I'm using pre-generated gamelist.xml files so the file extension in the system entries isn't used. es_systems.txt

"df" with the storage disk (/mnt/store) mounted:

Filesystem        Size  Used Avail Use% Mounted on
devtmpfs          7.7G     0  7.7G   0% /dev
tmpfs             7.7G     0  7.7G   0% /dev/shm
tmpfs             7.7G  1.2M  7.7G   1% /run
tmpfs             7.7G     0  7.7G   0% /sys/fs/cgroup
/dev/nvme0n1p2    466G  262G  203G  57% /
tmpfs             7.7G  8.0K  7.7G   1% /tmp
/dev/nvme0n1p1    511M   34M  478M   7% /boot
tmpfs             1.6G  4.0K  1.6G   1% /run/user/1000
tmpfs             1.6G     0  1.6G   0% /run/user/0
//10.0.0.2/Media   20T   17T  2.2T  89% /mnt/emerald
/dev/sda          5.5T  5.4T  140G  98% /mnt/store

"/etc/emulationstation/es_systems.cfg" isn't being used and thus doesn't exist.

[pjft]

Thanks.

Maybe it wasn’t clear, but my feedback is that it isn’t related to it checking for paths when loading the content, but rather when initializing the Platforms - which is not related to gamelists. Hence the errors “Unknown Platform for system XXX” when it fails to load, during the es_systems.cfg parsing. Gamelist-only does not check for any paths for the games whatsoever in itself, that you can trust.

I just don’t have a good idea as to why that is messing up the loading of the platforms section, and that’s what’s puzzling me. I was hoping that ES would be using a rogue es_systems.cfg file that would be causing errors somewhere.

Just for sanity checking, would you share your es_systems.cfg file with us?

[Enverex]

Already did, it's roughly in the middle of my last response (look for the blue link, it's not the most obvious though).

The "Unknown platform for system XXX" happens even when it does display correctly, so that may be a red herring.

[pjft]

Thanks, didn’t see the link as you were embedding everything else!

The Unknown platform for XXX seems to happen for all your systems in the bad load, namely the ones for which you do have gamelists for. In the good load it only seems to happen for a few odd ones that you likely don’t have games for. Do you confirm my assessment?

[Enverex]

Hah, sorry. The systems file was rather large so I attached it instead.

Originally I thought that was the case with the Unknowns, but I double checked the log for both the good and bad starts and it was identical (I did a diff to make sure I wasn't missing anything). AFAIK the "unknown platform" thing is harmless, it just means ES doesn't recognise the name of the platform for some of the features (metadata scanning, I'd guess?). All the very common platforms (e.g. gameboy, etc) don't show the unknown platform error, even in the bad load log seen above.

[pjft]

Hah. Thanks for double checking then - I suppose I was reading it wrong.

I’m traveling for work this week so won’t be able to even look into this, but to be honest without being able to replicate it on my end I will probably struggle to determine what may be happening. That is, if nobody else has any ideas on why that may be happening, as I don’t know a lot about this code.

I’ll try to look through the code and see what might be happening.

Would you be willing to share one sample gamelist as well? That PC games one is a good candidate. Also, could you see if for some weird reason the gamelist itself changes with and without things mounted? Symlinks and all?

Thanks.

[Enverex]

I think recreating the issue may actually be quite easy. Just create a platform/system that points to a non-existent location, make sure it has a gamelist with at least a few entries in it (which also point to somewhere in the non-existent location), and then start EmulationStation with the --gamelist-only option. Logically that seems like it should reproduce the problem.

Example gamelist.xml file (renamed to .txt else it won't let me attach): gamelist.txt

Gamelist doesn't appear to be modified by running in either the good or bad scenario, remains normal looking.

[pjft]

Got it, thanks.

The error will be here:

https://github.com/RetroPie/EmulationStation/blob/bf028191255c5f4948bd313d7facd09ba69bf5b6/es-app/src/Gamelist.cpp#L126

when we have gamelist-only set and the file can’t be found, which I suspect will end up here somehow:

https://github.com/RetroPie/EmulationStation/blob/bf028191255c5f4948bd313d7facd09ba69bf5b6/es-app/src/Gamelist.cpp#L71

so it may be either an issue with getStem on your OS or it might have been something that has been broken forever.