muldjord / skyscraper

Powerful and versatile game scraper written in c++
GNU General Public License v3.0
477 stars 127 forks source link

Add support for EmulationStation Desktop Edition #340

Open leonstyhre opened 2 years ago

leonstyhre commented 2 years ago

Describe the feature / request Could you please add support for EmulationStation Desktop Edition (ES-DE)? This is a fork of RetroPie EmulationStation that has evolved quite a lot with additional functionality and changes to core logic. For instance media asset links are no longer used in the gamelist.xml files but instead the media files simply need to match the game ROM names. Overall the media handling has been rewritten almost completely. ES-DE also supports a number of additional media types compared to older forks like RetroPie ES.

The following directories are used for game media:

All are kept inside a media directory whose path is configurable, leading to a fully portable setup (it defaults to ~/.emulationstation/downloaded_media).

Additional context The user guide contains detailed information about ES-DE and if you have further questions or need more information I'm very happy to provide that (I'm the maintainer of the project).

https://gitlab.com/leonstyhre/emulationstation-de/-/blob/master/USERGUIDE.md

This is the website for the project: https://es-de.org

naushir commented 2 years ago

@leonstyhre out of curiosity, would it be possible to change ES-DE to allow keeping the artwork in the ./roms//media directory and parse the locations through gaimelist.xml as is done on RetroPie's EmulationStation?

leonstyhre commented 2 years ago

@naushir Removing media links from the gamelist.xml files was one of the first thing I did when starting to work on ES-DE and it's a large improvement over RetroPie EmulationStation. This change makes media in ES-DE completely portable and a user can replace a media file simply by dropping a new file in place without having to update any XML files.

The media directory can also be relocated using a menu option so it's really fully portable. It also makes the application perform better as gamelist parsing is faster without having to load media information and check whether the files are actually present on the filesystem.

So no, this will not be reverted back to the old logic :)

naushir commented 2 years ago

Got it! Thanks for the summary.

Crono141 commented 1 year ago

so, I'm just starting trying to use Skyscraper with ES-DE, and it doesn't seem to drop the assets into the correct place. I'm not sure if I'm doing something wrong, but I just spent 2 days scraping, with the stdout seeming to put things where they are supposed to go, but after checking later none of the metadata or assets went in those locations, and appear to be completely lost. IS there a methodology for getting Skyscraper data into ES-DE?

jdalmanza commented 1 year ago

I'm not sure how different the DE version is but it should still read the scrape data of it's in the roms directory did you use the "relative" option when making the game list?

Also, and it's possible I could be reading this wrong but the -o option for output is for the cache location not the actual media location. You have to run skyscraper twice, once to build the cache with assets and once again to create the gamelist.xml did you skip that step?

Crono141 commented 1 year ago

I see. I did indeed forget to build the gamelist. I didn't realize this was necessary.

jdalmanza commented 1 year ago

Run it again without the -s option ... But, iif you redirected the cache to a different location you might need to adjust your command line to account for this location

Crono141 commented 1 year ago

So I've updated the screenscraper INI with I think these 2 key points: mediaFolder="/home/arcade/Emulation/tools/downloaded_media" cacheFolder="/home/arcade/.skyscraper/cache" but none of the media ended up in the media folder, but the cache folder was full of the assets with (I think) hashed names. This led me to believe things were broken, as I'd done half a scrape through the ES-DE UI and those assets were showing up just fine, but nothing from skyscraper was where I was expecting it. Does Skyscraper save all the media in the cache folder, and then the gamelists just references that location? As opposed to ES-DE which is looking for same name metadata as outline in this issues OP.

jdalmanza commented 1 year ago

It will make a copy by default, I think there is an option to use symlinks but I'm not positive I'll have to take a look at the options again.

So it sounds to me like you have the cache in the same location as the roms and maybe some scraped media from ESs internal scraper?

You could move the cache to another more convenient location it will probably save you some headaches when making your game list

Crono141 commented 1 year ago

No, the cache is actually on a separate drive in the .skyscraper folder (default behavior). The roms are in Emulation/roms. I'm executing skyscraper with this command line, to read the roms folder and call skyscraper for each folder.

ls -1 /mnt/storage/Emulation/roms | while read line ; do Skyscraper -f emulationstation -s screenscraper -u myuser:mypass -p $line | tee /home/arcade/.skyscraper/logs/$line.log ; done

So my Emulation folder in my home folder is symlinked from /mnt/storage/Emulation. I iterate through all folders in my roms folder so I can grab everything in one go. ES-DE looks in Emulation/tools/downloaded_media for its assets. So that is why I was confused, as the output from skyscraper seemed to indicate it was putting the assets in that place, but nothing ever ended up there after it finished. So do I understand that it doesn't actually put the assets there until you call skyscraper without the '-s' option? And then it pulls the cache data into that location? Why not just download the data to the target folder from the getgo and skip creating the gamelists? I think that is the purpose of the original request, to add this behavior into skyscraper for ES-DE.

jdalmanza commented 1 year ago

yeah, i'm not the dev, so I can't comment on that but i think the idea is scrape once, create many game lists. Also i think many people overlook the functionality of the artwork.xml and all those options which imo is the real power of skyscraper. Personally, I have a ginormous skyscraper cache on an external drive and that affords me the ability to create new playlists on new devices pretty much whenever i want and not have to scrape again, especially when there are really restrictive limits on most scrape sources

Crono141 commented 1 year ago

Interesting. I hadn't thought of that use case. And in that case it makes a lot of sense.