ScummVM Scraping Information

[Enverex]

Describe the bug Not sure whether this is a request for documentation or a report about a scraping issue so I'll keep the description rather option: Trying to scrape with the platform "ScummVM" doesn't seem to really yield any results. The files it's scraping are named as per the expected title, sans any brackets. So "Blade Runner (CD Windows)" will be "Blade Runner.sh". It's finding the files to lookup, but finds no matches when checking against Screenscraper.

To Reproduce Scrape a folder of files with the name of the game. Scraper will show the correct game names in the lookups, but find no results. This is against the SCREENSCRAPER source (which seems to have the games filed under DOS or Windows on their site). A good example (as it doesn't really exist on any other platforms) is Toonstruck. Searching for that on ScreenScraper.fr finds the game explicitly under the ScummVM category, but Skyscraper with the Screenscraper source finds nothing.

Special circumstances I'm honestly not sure what format it's expecting here. For my own scraper (i.e. when not using Skyscraper) I let it use the top level directories as the "game" and it searches based on those names. For Skyscraper it appears that it has to be files, so I've created .sh (Linux shell scripts) for each game with the name of the game which in turn just run scummvm with the relevant commands (I've added .sh to be an included format for every platform via the Skyscraper config file). I've seen mention of having to use .scummvm files for this platform though? If that's the case, do those files have special contents which is used for the scraping process? (if so, what? This was the bit where I mentioned at the start that if it uses something odd like this, there may be missing documentation).

Terminal output Examples:

Platform:           'scummvm'
Scraping module:    'screenscraper'
Input folder:       '/emu/games/ScummVM'
Game list folder:   '/home/enverex/.emulationstation/gamelists/scummvm'
Covers folder:      '/home/enverex/.emulationstation/gamelists/scummvm/media/covers'
Screenshots folder: '/home/enverex/.emulationstation/gamelists/scummvm/media/screenshots'
Wheels folder:      '/home/enverex/.emulationstation/gamelists/scummvm/media/wheels'
Marquees folder:    '/home/enverex/.emulationstation/gamelists/scummvm/media/marquees'
Local db folder:    'dbs/scummvm'

[trimmed]

Starting scraping run on 259 files using 6 threads.

#1/259 (T4) Pass 1 Pass 2 Pass 3 Pass 4 Pass 5 ---- Game 'Amazon - Guardians of Eden' not found :( ----

0/1/1
Elapsed time   : 00:00:01
Est. time left : 00:08:08

#2/259 (T6) Pass 1 Pass 2 Pass 3 Pass 4 Pass 5 ---- Game 'Backyard Baseball 2003' not found :( ----

0/2/2
Elapsed time   : 00:00:01
Est. time left : 00:04:15

#3/259 (T3) Pass 1 Pass 2 Pass 3 Pass 4 Pass 5 ---- Game 'Backyard Baseball' not found :( ----

0/3/3
Elapsed time   : 00:00:02
Est. time left : 00:02:50

#4/259 (T1) Pass 1 Pass 2 Pass 3 Pass 4 Pass 5 ---- Game '3 Skulls of the Toltecs' not found :( ----

0/4/4
Elapsed time   : 00:00:02
Est. time left : 00:02:12

#5/259 (T5) Pass 1 Pass 2 Pass 3 Pass 4 Pass 5 ---- Game 'Astro Chicken' not found :( ----

0/5/5
Elapsed time   : 00:00:02
Est. time left : 00:01:46

#6/259 (T2) Pass 1 Pass 2 Pass 3 Pass 4 Pass 5 ---- Game 'A.J.'s World of Discovery' not found :( ----

Technical information

• Skyscraper version: v2.9.5
• Internet connection: VDSL
• OS: Arch Linux x86_64
• OS Version: N/A

[muldjord]

Why do you want to use .sh files? I don't understand that choice as RetroPie already has builtin .svm and .scummvm support that launches the games automatically (should work when using it on Arch aswell). Using .sh is probably not in the screenscraper database, which is why it doesn't provide a result.

Creating the .svm files is easy. Just start ScummVM, add a game, point to the folder where the game resides and save the config using the default name. Then it'll work. It will even work well with thegamesdb and mobygames if you use the official file endings, as Skyscraper has seemless integration with the scummvm.ini config file which allows the weird naming to be translated to something useful when scraping using a search based scraping module such as thegamesdb.

[Enverex]

The sh files were just a means to an end. They were just scripts that launch the respective game (I set EmulationStation to just launch that system with "bash" so it runs the script which in turn runs the game).

I'm not using RetroPi, I'm using Skyscraper as an replacement of the inbuilt scraper of EmulationStation on an x86_64 Linux PC.

Adding the games to ScummVM's own interface doesn't have a save option, it just creates a single ~/.config/scummvm/scummvm.ini file with information about all the games that have been added through the interface.

Where do these "svm" files come from? As I've not seen one before and ScummVM itself doesn't seem to create them.

[muldjord]

The .svm or .scummvm files are just empty files that have the same name as the square bracket name inside of scummvm.ini. It tells RetroPie which game to launch when running the scummvm start script.

But you can easily scrape your .sh files aswell. Read below.

[muldjord]

You can also use the new aliasMap.csv file to create aliases for your filenames. The file is located at ~/.skyscraper/aliasMap.csv. That way you can create a file called fullthrottle.sh and add a line into the file with fullthrottle;Full Throttle. Then it will use Full Throttle when searching for the game when using thegamesdb or mobygames.

Do this for all your games, then scrape them like this:

$ Skyscraper -p scummvm -s thegamesdb --refresh
$ Skyscraper -p scummvm -s mobygames --refresh
$ Skyscraper -p scummvm

The last command will scrape all of your games / sh files by combining the cached data from both sources.

[Enverex]

Possibly a silly question then, but why name the script fullthrottle.sh and use a the lookup table rather than just naming the original shell script "Full Throttle.sh"? (assuming I've told the es_systems file how to handle the script files correctly).

Is the ScummVM short-name actually used in the lookup process?

Also from what you've described, I assume it's pretty much impossible to use Screenscraper for ScummVM in general due to the fact it wants a hash of the game's executable to look up?

[muldjord]

I only suggested using the aliasMap.csv as your original question was about having issues with scraping. I assumed you had tried other scraping modules beside screenscraper with no luck. And some people (including myself) are quite religious about their filenames.

No, it can use screenscraper just fine as it also looks up based on exact filename. But it probably only has the .svm and .scummvm filenames in there. You can check this yourself. Also, it's not the games executable that's looked up. It's whatever you tell it to look up. It will always do a checksum search and an exact filename search. In this case on your sh scripts. Where obviously the checksum based search will fail. But the filename search might work.

[Enverex]

Just to clarify, "exact filename search" includes the file extension? I think that's where my confusion comes from. I was expecting it to submit the name "Full Throttle" and the platform "ScummVM". I wasn't expecting it to be searching for "Full Throttle.sh". So that explains that issue if that's the case.

[muldjord]

Yes, screenscraper searches for the entire filename including the suffix. This is not the case for the other modules though.