[ES 2] Additional Scrapers

[nemo93]

1. It would be great to be able to add more "scraping databases".
2. Also to be able to select more than one scraping DB to have more results would be good.
3. to be able to pause/resume scraping session as you have to start all over again after you suffer from a crash or because you had to shutdown ES.
4. to be able to get back to a previously scraped game during a scraping session
5. fully automated scraping - no user intervention

[Aloshi]

1. Yeah, it would. The code supports this. Someone just has to write more scrapers.
2. The code sort of supports this (you can mix and match multiple scraper requests, even from multiple sources, and it will fill the result list until it runs out of queries or hits a maximum of 7 results). TheArchive scraper barely works so I haven't really bothered, but in the future it should be possible to set an "order" for scrapers.
3. Set the filter to "games without images" and ES will do this.
4. Well, at the very least you can still re-scrape from the game list right now.
5. Turn off "User Decides Conflicts." Your results will be wildly inaccurate. I couldn't find any database that supports anything like CRC matches to solve this problem.

[nemo93]

1. great! how to write scrapers? also only scraper from thegamesdb return good results/high matches. Thearchive scraper is almost never working as you said. Final thought: it could be good to have a scraper querying wikipedia as lot of infos there.
2. perfect.
3. in fact, I already have gamelist.xml with images but no description or whatever metadata. To have more filters (games without description, without publisher, etc) would help a lot.
4. silly me :)
5. ok

Let me add a final item related to the location where images are saved. Currently it's within the ".emulationstation/downloaded_images/" folder. Problem is that it's not a shared drive and to modify/change images I have to use FTP :( If the default directory for downloaded images could be the same as to where the roms are this would be awesome.

[nilsbyte]

to 2. I would suggest an additional entry in the scraper settings menu. it should look like

PRIMARY SCRAPER < THEGAMESDB / ARCHIVE.VG / ... > ALTERNATE SCRAPER < NONE / THEGAMESDB / ARCHIVE.VG / ... >

my two cents as the UI designer of this project.

[Aloshi]

Scrapers are written in C++. All the relevant code can be found here, I don't have a tutorial for writing one.

I opted to keep downloaded images in the ~/.emulationstation folder because I know some people don't like ES-specific files cluttering their games folder (myself included). If you want to move the folder to a shared drive, move the downloaded_images folder to your shared drive, then use the ln utility to create a link with something like ln --symbolic /mydrive/downloaded_images ~/.emulationstation/downloaded_images.

[Maikeximu]

Just a quick update. I tried to create a symbolic link "If you want to move the folder to a shared drive, move the downloaded_images folder to your shared drive, then use the ln utility to create a link with something like ln --symbolic /mydrive/downloaded_images ~/.emulationstation/downloaded_images." but the scraper wants to "mkdir" the folder so it seems that it doesn't consider it as a hard link.