search

Universal-Rom-Tools / Universal-XML-Scraper

Scraper de Rom
195 stars 46 forks source link

Not Scraping ALL roms #135

Closed MaryBray140 closed 7 years ago

MaryBray140 commented 7 years ago

Hello Everyone, I'm having some issues with the Scraper. I'm currently running RetroPie 4.1.9 and the latest release of the XML Scraper (2.1.0.2). Before the update, a week ago I scraped the N64 games I have and only 49 out of 185 games/roms were "found". I didn't worry about it too much because I was getting ready to reinstall RetroPie on a new (upgraded) SD card. Last night I got everything completed with the general setup and configured somethings including the HotKeys for Mupen64Plus. Leaving the XML Scraper last on my list for the N64 roms. I ran the Scraper for N64 on the new SD card and ended up with the same result of only 49 out of 185 games scraped. I do also have the list of "Nintendo_64_missing" that was dumped in the roms/n64 file provided after the Scraper was finished.

Questions:

  1. Do you or anyone here have some troubleshooting to offer?

  2. What can i do with the Nintendo_64_missing file, am I to submit it to the developers somehow?

BSAKat commented 7 years ago

I'm having a similar problem with NES and SNES images. Like, for some reason it won't download the images for Zelda II or Metroid.

MaryBray140 commented 7 years ago

Yea, same thing here too; a lot of popular titles were not being "found". Within less than 24hrs my issues have been resolved. So, I ended up back tracking to the source (https://www.youtube.com/watch?v=0ZexFv5QdM0) of where I first discovered this Scraper. I was instructed to go to the main website (http://screenscraper.fr/) and click on the link on the top "Left" corner of the Main Page called "Chat IRC". Go there, log-on and ask the developers there (politely) if they could assist you in adding your missing roms to their database. I did all of my communication with a person named "Marble". He assisted me by adding my missing roms to their database. What you must do first is while your Raspberry Pi is hooked up to your Network, open up File Explorer in Windows and go to RetroPie/roms/GAME SYSTEM/missing_files. Then while on the Chat IRC click on the "Upload to paste Bin" tab on the lower "Right" hand side. Click and drag the missing_files to the box and it will create a link that you can post on the chat for them to easily access for review.

Note: Be patient while on there. You may or may not get a quick response. They're based out of Europe and you may find as I did that there is a bit of a language barrier.

BSAKat commented 7 years ago

Oh, wow. Thanks for the heads up! I'll do all that right now.

MaryBray140 commented 7 years ago

My pleasure!

MaryBray140 commented 7 years ago

I see you in the Chat Room now! Everything looks proper and your link is working correctly. Again, be patient. They're probably just waking up. It's approx 8:30am in Europe. Good luck

Universal-Rom-Tools commented 7 years ago

I'm in this chat room too (screech or screech_taf) ;) but on French time ;)

And you can pastebin here your missing ;) I'll import them to the DB ;)

MaryBray140 commented 7 years ago

Oh great! Thank you very much!

BSAKat commented 7 years ago

Awesome. Thanks, screech! Here's my NES and SNES missing files. _NES_missing.txt _Super Nintendo_missing.txt

MaryBray140 commented 7 years ago

Hello Screech, thank you again in advance for all of your help for making this the best scraper available for RetroPie.

Here are my missing_files: _NES_missing.txt

_Super Nintendo_missing.txt

_Nintendo 64_missing.txt

_Master System_missing.txt

_Megadrive_missing.txt

Thank you

Universal-Rom-Tools commented 7 years ago

@MaryBray140 Imported all your CRC... Some already Exist (but are orpheans), Some are autoassociated (so it's already OK), and Some are just Orpheans (waiting for an association to his game ;) )

@BSAKat I have a problem about the "name" of your rom... I think you rename them. But we miss an information (the country tag like (U) or (USA) ). So it's too bad for put them in database (we can't choose the Rom country and we cannot match the "good" metadata and media corresponding :( )

MaryBray140 commented 7 years ago

Great! Thank you very much Screech!

images

MaryBray140 commented 7 years ago

@BSAKat, I encountered the same issue a few weeks back. I hated the title layout of the emulators not knowing what the symbols at the end of a title meant at the time. I renamed them not realizing that once scraped, the symbols would then be hidden from the titles on the emulationstation front-end. I would suggest re-downloading the "missing" emulators and then try again with the scraper; perhape to find that they're in the DB after all.

Lastly, check out the link below. Scroll down on the page and It'll help you to identify the symbols.

https://en.wikipedia.org/wiki/GoodTools

BSAKat commented 7 years ago

Oh! Huh, right on. Yeah, I renamed all my ROMs. I'll just download replacements for the ones I'm missing and see if that helps.

BSAKat commented 7 years ago

That fixed it! I just re-downloaded my ROMs, left the tags on them, and scraper found them no problem. Thanks again @MaryBray140 and Screech!

MaryBray140 commented 7 years ago

you're welcome

MaryBray140 commented 7 years ago

@BSAKat I would also like to encourage you to register and make a donation to the site (if you haven't already) at http://screenscraper.fr/. They've definitely earned it; with all of the tedious work that it requires to assemble a service like this.

P.S. Best of luck with any future projects.

BSAKat commented 7 years ago

Way ahead of you! I registered and backed them a couple days ago. They do really good work.

Universal-Rom-Tools commented 7 years ago

Thank you for your support ^^

FYI : If you are registered to Screenscraper and make donation or "just" contribute to increase Data in the DB. You can put your log/pswd in the general config ;) It will grant you some "threads" to greatly increase scraping time ;)

(Thread are simultaneous Scrape engine launch at the same time. So 2 threads nearly mean 2x less time to scrape ;) )

I close the Issue.

and don't hesitate to submit here new missing file to increase DB matching CRC ;)