Missing game with "&" character

[magostinelli]

For example the rom "North & South" in nes is missing inside the emulation station menu. If i rename the rom, it's working.

You can view some detail here: https://retropie.org.uk/forum/topic/11350/missing-game-inside-emulation-station

[csonsino]

I don't see this behavior. All of the games with a & character in the name or filename appear correctly for me. I would note that in the gamelist.xml tags, the & is replaced by & (<path>, <name>, <image>,...), for example <path>./North &amp; South.zip</path> instead of <path>./North & South.zip</path>.

Maybe it's a locale or other system configuration issue?

[magostinelli]

Hello, I have this locale (I'm italian, I suppose this is default in retropie):

LANG=en_GB.UTF-8 LANGUAGE= LC_CTYPE="en_GB.UTF-8" LC_NUMERIC="en_GB.UTF-8" LC_TIME="en_GB.UTF-8" LC_COLLATE="en_GB.UTF-8" LC_MONETARY="en_GB.UTF-8" LC_MESSAGES="en_GB.UTF-8" LC_PAPER="en_GB.UTF-8" LC_NAME="en_GB.UTF-8" LC_ADDRESS="en_GB.UTF-8" LC_TELEPHONE="en_GB.UTF-8" LC_MEASUREMENT="en_GB.UTF-8" LC_IDENTIFICATION="en_GB.UTF-8" LC_ALL=

I confirm that in my gamefile.xml & is replaced by &: pi@retropie:~ $ grep "&" ./RetroPie/roms/nes/gamelist.xml | wc -l 122

pi@retropie:~ $ grep "&" ./RetroPie/roms/nes/gamelist.xml | wc -l 122

[csonsino]

I'm in the US and use "en_US.UTF-8", but I reconfigured my pi to use "en_GB.UTF-8" (my locale output looks the same as yours now) and it still works. Are you all up-to-date on your Raspbian/RetroPie installation (have you re-run the retropie_setup.sh and updated everything recently)? Are you using the default "Carbon" theme, or a different/custom one?

[magostinelli]

Yes, it's all up-to-date. I use the default theme.

I used sselph scraper, we can try to regenerate the gamelist.xml, which procedure you suggest me?

[csonsino]

I don't think that it's a problem with the scraper or gamelist.xml file. It seems like the scraper is finding the file (otherwise it wouldn't even make it into the gamelist.xml), and it also seems like it's generating the proper XML (escaping the & with &).

It seems like the problem might be in the emulationstation, perhaps while it's iterating through the gamelist XML nodes. I'll take a look in the code to see if I can find anything, but we may have to build you a specially instrumented emulationstation binary with additional debugging.

Have you looked in ~/.emulationstation/es_log.txt to see if there are any errors being logged?

[magostinelli]

Hello, inside the log the only error is some missing gamelist.xml, because I don't have the roms for all the emulator.

[csonsino]

I'm going to create a debug branch for this, and a few additional details would help. At this point, I think that the most helpful information is trying to figure out if this applies to a certain gamelist view or all of them.

1. Which GAMELIST VIEW STYLE do you use? You can find it in the Menu in UI SETTINGS towards the bottom. (It should be set to AUTOMATIC unless you changed it)

2. Do you have the video files for the games in your gamelist.xml, and the gamelist.xml entries have the <video> tags pointing to them? If the GAMELIST VIEW STYLE is set to AUTOMATIC and the video files are downloaded, then emulationstation will pick the VIDEO gamelist view instead of the DETAILED gamelist view.

3. Does the game with the & show up in the list when you change the GAMELIST VIEW STYLE to BASIC? Change the setting back to AUTOMATIC for the next test.

4. This should actually cause the same behavior as using the BASIC setting in the last test, but it could provide some additonal hints. When you remove the gamelist.xml file, does the game with the & show up in the list? You can exit emulationstation (F4 on the keyboard), change the name of ~/.emulationstation/gamelists/nes/gamelist.xml and then either restart emulationstation or reboot. If you're using the AUTOMATIC setting for GAMELIST VIEW STYLE, it should then use the BASIC gamelist view instead of the DETAILED or VIDEO.

I'll start working on adding some debugging code, but if you could provide the details above then hopefully I can narrow down the problem area faster.

Thanks!

[magostinelli]

Hello, I'm out of house for holidays, I will reply to you when I come back (next week).

[magostinelli]

consino: I have done the test:

1. GAMELIST VIEW is set to AUTOMATIC

2. I haven't any VIDEO entry inside the gamelist.xml

3. No, not showing up changing view style.

4. I do the test, but the game is not showing up.

Thanks!

[csonsino]

I created a custom version of EmulationStation with additional debugging.

I posted a precompiled Raspberry Pi 3 binary @ https://filebin.ca/3VHY4fUQxj4i/emulationstation Or you can compile it yourself by cloning the source from my repo and following the normal EmulationStation build instructions:

git clone --recursive -b gamelist_debug https://github.com/csonsino/EmulationStation.git

In order to get the additional debug info, you need to launch the emulationstation binary with the --debug argument. So if you copied the binary to your /home/pi directory, you would cd there (or if you cloned the source into /home/pi/EmulationStation and built it yourself, cd to /home/pi/EmulationStation) and run ./emulationstation --debug

After launching the emulationstation binary with the debug flag, more information should be logged to ~/.emulationstation/es_log.txt. All of my extra logging starts with ***, so it should be easy to spot.

You should see a line like:

lvl3: *** SystemData::populateFolder(), system: nes

followed by all of the games found for the nes system.

You should also see a few lines for the game as the gamelist.xml is parsed:

lvl3:   *** Gamelist::parseGamelist(), defaultName: "North & South (U) [!]"
lvl3:   *** Gamelist::parseGamelist(), Using name: "North and South (USA)"
lvl3:   *** Gamelist::parseGamelist(), Game added

Finally, you should see a line when the game is added to the BasicGameListView (which is also used for the DetailedGameListView):

lvl3: *** BasicGameListView::populateList(), Adding game to list: North and South (USA)

If any of those log lines are missing, then we should be able to start figuring out where the problem is happening. If you see all of those log lines, then I still have no idea why the game is not showing up and I'll have to look at all of the code again.

I'd be happy to look at your whole log file, but I'm not sure what the rules are in terms of posting log files that include details on what roms you have on your system.

[hex007]

@magostinelli This is not EmulationStation bug but rather sselph scrapper error. ES puts & in the gamelist rather thanjust & which you have in your gamelist posted on the forum.

If you want to test it out, manually delete the entry for the game from gamelist and play game once. ES will automatically add that entry with & in the game name and location while updating the play count.

Test file name : te & st.gb

<path>./te &amp; st.gb</path>
<name>te &amp; st</name>
<playcount>1</playcount>
<lastplayed>XXXXXXXXXXXXXXXX</lastplayed>

If this addresses your concern, please close the issue

[magostinelli]

@hex007 I have & in my gamelist.xml. I deleted the entry inside gamelist.xml but game is not shown, I also have a copy of the rom without & in the name that display correctly.

Now I will do the csonsino test, for me is not a problem the send all the log, I can send you in a private message?

[magostinelli]

I post here an extract of the file,:

...
lvl3:   *** SystemData::populateFolder(), Added game for file: North & South
lvl3:   *** SystemData::populateFolder(), Added game for file: Ikari III - The Rescue
lvl3:   *** SystemData::populateFolder(), Added game for file: Wheel of Fortune - Family Edition
lvl3:   *** SystemData::populateFolder(), Added game for file: Pipe Dream
lvl3:   *** SystemData::populateFolder(), Added game for file: Laser Invasion
lvl3:   *** SystemData::populateFolder(), Added game for file: North South
lvl3:   *** SystemData::populateFolder(), Added game for file: Tiles of Fate
lvl3:   *** SystemData::populateFolder(), Added game for file: Roundball - 2-on-2 Challenge
lvl3:   *** SystemData::populateFolder(), Added game for file: Bigfoot
lvl3:   *** SystemData::populateFolder(), Added game for file: Jeopardy! - Junior Edition
lvl3:   *** SystemData::populateFolder(), Added game for file: Ultima - Quest of the Avatar
lvl3:   *** SystemData::populateFolder(), Added game for file: Peek-A-Boo Poker
lvl3:   *** SystemData::populateFolder(), Added game for file: Huge Insect
lvl3:   *** SystemData::populateFolder(), Added game for file: Fantasy Zone
lvl3:   *** SystemData::populateFolder(), Added game for file: Kings of the Beach
lvl3:   *** SystemData::populateFolder(), system: nes
lvl2:   Couldn't find entry in index! 1
lvl2:   Couldn't find entry in index! FALSE
...
lvl3:   *** Gamelist::parseGamelist(), defaultName: "Nobunaga's Ambition"
lvl3:   *** Gamelist::parseGamelist(), Using name: "Nobunaga's Ambition"
lvl3:   *** Gamelist::parseGamelist(), Game added
lvl3:   *** Gamelist::parseGamelist(), defaultName: "North & South"
lvl3:   *** Gamelist::parseGamelist(), Using name: "Les Tuniques Bleues - North & South"
lvl3:   *** Gamelist::parseGamelist(), Game added
...
lvl3:   *** BasicGameListView::populateList(), Adding game to list: North South

I have a copy of the game in my rom folder:

pi@retropie:~/RetroPie/roms/nes $ ls -l North*
-rwxrwxrwx 1 pi pi 111468 Dec 24  2016 North & South.zip
-rwxr-xr-x 1 pi pi 111468 Jul  4 18:44 North South.zip

[hex007]

@magostinelli I would like to see the game list where it is affected and your emulation station settings file (~/.emulationstation/es_settings.cfg).

[magostinelli]

I uploaded the two files here: https://transfer.sh/dUHgw/es_settings.cfg https://transfer.sh/HuO7D/gamelist.xml

[hex007]

I tested with a sample game from list and it works for me. Will need some time to debug this

[magostinelli]

Hello, can I make some other test to solve the issue?

If doesn't affect the bug, I want to change my locale to Italian.

[hex007]

@magostinelli Can you update to the latest binary and test it with that?

I dont think changing Locale should affect anything. At most you might need to revert back.

[csonsino]

I also think that it should be fine to change the Locale.

I thought that the last log message: lvl3: *** BasicGameListView::populateList(), Adding game to list: North South meant that the game was being added to the list for display, but apparently there is something else going on. I'll have to keep looking.

[magostinelli]

@csonsino the game was added because I have two copy (one with &, one without) of the rom files see in my previous messages. I will try updating to the last version.

[csonsino]

Oh, I missed that you have 2 copies of the game. It looks like the North & South version is using the name Les Tuniques Bleues - North & South. Do you have a line in that log that says: lvl3: *** BasicGameListView::populateList(), Adding game to list: Les Tuniques Bleues - North & South, and is the game showing up in your list with that name?

[magostinelli]

It Works! I didn't notice this line in the log.

Who have changed the name of the game? The scraper?

I googled for "Les Tuniques Bleues" and i found that it was a french name.

[hex007]

@magostinelli Please close the issue if it has been solved. Thanks.

[magostinelli]

@hex007 It will be solved when the game are listed in emulationstation with the correct name.

[pjft]

@magostinelli I'm not sure I get that comment. The scraper might have matched that ROM to that name, depending on the database entry it selected for that ROM. If you want it to be listed with a different name, edit the gamelist.

If the gamelist shows "Les Tuniques Bleues - North & South" and ES shows that entry, then that is the correct name, and this is not an issue. ES has no way to guess what name it should be other than the info on the gamelist.

Please let us know what's the exact state, and close - or keep open - accordingly. But it doesn't seem to be like it's not showing the "&" character, as stated, and it also doesn't seem to be hiding any particular game now.

[magostinelli]

I prefer to keep open the issue, to complety solve the problem for me, but also for other retropie users. I don't have the name the name "Les Tuniques Bleues - North & South" my gamelist.xml, can I readd it or relaunch the scraper?

I had a backup: `diff gamelist.xml gamelist.xml.old 5060a5061,5072

  <game id="1979" source="theGamesDB.net">
      <path>./North &amp; South.zip</path>
      <name>North and South</name>
      <desc>Chaaarge!!! The bugle is calling you to the battle field, and your troops are awaiting your orders. Demonstrate your brilliant strategy skills: move entire armies, surround the enemy, attack forts, intercept gold laden trains, and thus re-write the history of the American Civil War!</desc>
      <image>~/.emulationstation/downloaded_images/nes/North &amp; South-image.jpg</image>
      <rating>0.6</rating>
      <releasedate>19900921T000000</releasedate>
      <developer>Seika</developer>
      <publisher>Kemco</publisher>
      <genre>Strategy</genre>
      <players>2</players>
  </game>

8788c8800 <

`

[pjft]

Can you share your gamelist.xml file from ~/RetroPie/Roms/nes and from ~/.emulationstation/gamelists/nes via pastebin or some other way? On Wed, 23 Aug 2017 at 13:52 Michele Agostinelli notifications@github.com wrote:

I prefer to keep open the issue, to complety solve the problem for me, but also for other retropie users. I don't have the name the name "Les Tuniques Bleues - North & South" my gamelist.xml, can I readd it or relaunch the scraper?

I had a backup: `diff gamelist.xml gamelist.xml.old 5060a5061,5072

./North & South.zip North and South Chaaarge!!! The bugle is calling you to the battle field, and your troops are awaiting your orders. Demonstrate your brilliant strategy skills: move entire armies, surround the enemy, attack forts, intercept gold laden trains, and thus re-write the history of the American Civil War! ~/.emulationstation/downloaded_images/nes/North & South-image.jpg 0.6 19900921T000000 Seika Kemco Strategy 2

8788c8800 <

`

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/RetroPie/EmulationStation/issues/174#issuecomment-324320028, or mute the thread https://github.com/notifications/unsubscribe-auth/AVAV7eBQDE49n-qMcgzG2QlLkpXD6Vm_ks5sbCCegaJpZM4OOLCj .

[magostinelli]

I don't know that there are two gamelist.xml!

Inside /home/pi/RetroPie/roms/nes, i have:

`

./North & South.zip

    <name>Les Tuniques Bleues - North &amp; South</name>
    <desc>Based on the Belgian comic book, "Les Tuniques Bleues", this strategy game allows you to replay the American Civil War from both the North and the South sides.

In the strategy part of the game, you can move your troops on a map of the US. If two opposing armies enter the same state, the battle is played out in an action sequence. Additional action sequences must be accomplished when capturing an enemy fort or train.

The game also offers a two-player-mode.

./downloaded_images/North & South-image.jpg

    <thumbnail/>
    <rating>0.8</rating>
    <releasedate>00000000T000000</releasedate>
    <developer>Kemco</developer>
    <publisher>Infogrames</publisher>
    <genre>Strategy</genre>
    <players>1-2</players>

` /home/pi/RetroPie/roms/nes/gamelist.xml https://transfer.sh/lsgjM/gamelist.xml

/home/pi/.emulationstation/gamelists/nes/gamelist.xml: https://transfer.sh/13zZUn/gamelist.xml

[csonsino]

Well I'm glad that the game is showing up for you, even if it has an alternative name. I would suspect that the scraper source is the one supplying that alternate name. I've only used the scraper a few times, but I think I saw an option to change the source database/site. Maybe you should try choosing a different one?

[pjft]

Look, in the gamelist you sent from /home/pi/RetroPie/roms/nes/gamelist.xml you have this:

<game id="1476" source="ScreenScraper">
    <path>./North &amp; South.zip</path>
    <name>Les Tuniques Bleues - North &amp; South</name>
    <desc>Based on the Belgian comic book, "Les Tuniques Bleues", this strategy game allows you to replay the American Civil War from both the North and the South sides.

In the strategy part of the game, you can move your troops on a map of the US. If two opposing armies enter the same state, the battle is played out in an action sequence. Additional action sequences must be accomplished when capturing an enemy fort or train.

The game also offers a two-player-mode.
</desc>
    <image>./downloaded_images/North &amp; South-image.jpg</image>
    <thumbnail/>
    <rating>0.8</rating>
    <releasedate>00000000T000000</releasedate>
    <developer>Kemco</developer>
    <publisher>Infogrames</publisher>
    <genre>Strategy</genre>
    <players>1-2</players>
</game>

That's where it's getting the name from. Please edit it or change it if you don't like that name.

Please close this issue.