search

Universal-Rom-Tools / Universal-XML-Scraper

Scraper de Rom
195 stars 46 forks source link

Missing File #159

Closed Universal-Rom-Tools closed 6 years ago

Universal-Rom-Tools commented 7 years ago

If you want to add some new Rom to the screenscraper DB,

just read this : https://github.com/Universal-Rom-Tools/Universal-XML-Scraper/wiki/Rom-Missing-on-Screenscraper

PhlynnStarwind commented 7 years ago

_Megadrive_missing.txt

PhlynnStarwind commented 7 years ago

_Nintendo DS_missing.txt

stefancallear commented 7 years ago

_Super Nintendo_missing.txt _Neo-Geo_missing.txt _Mame_missing.txt

Excellent project, he's a copy of sets that had missing data. Rom sets sourced from arcade punks. thank you

stefancallear commented 7 years ago

_Master System_missing.txt

Another rom set, this one had a lot of issues.

MajorDangerNine commented 7 years ago

I tried my best to rename my games and keep these lists short, but no altering of the names would scrape these last few games.

_Family Computer Disk System_missing.txt _Game Boy Advance_missing.txt _Game Boy_missing.txt _Mega-CD_missing.txt _PC Engine_missing.txt _Satellaview_missing.txt

All CHD compressed games are either Redump-verified or from a common source.

And in case you didn't know, CHD does the same thing as RomVault/Trrntzip.NET, in that the files are compressed exactly the same way each time, and goes a step further by removing all non-essential data, such as the possibly varying names for each BIN in a CUE, which results in the same CRC and MD5 for differently named sources as long as the game files are the same content-wise and the CUE is properly ordered.

Here are my CRC+MD5 scanned CHD libraries, which I hope are named well; TurboGrafx-CD doesn't have a lot of USA Redump-verified, so they had to be found from common sources, which I cleaned up the names on, and the same goes for a few PSX and Sega CD.

Sega CD - Mega CD (CHD Library).log TurboGrafx-CD - PC Engine CD (CHD Library).log PSX (CHD Library).log

I expect you'll want to look into CHD more to make sure I'm right. I actually haven't read up on it, just tested and found that the resulting files matched up content-wise, even when I purposely altered a BIN name.

ghost commented 6 years ago

Hi everyone, I too have several missing.txt entries for several consoles. Last time this topic was updated by the admin was in May. I think he's not adding them anymore.

I think Universal XML Scraper should have an update so we could do the missing.txt roms in manual mode. The manual mode would be integrated to the software making it easy for us to choose the correct version.

My filenames are only the rom name and I want to keep it clean like that. I won't change my whole naming convention for few missing entries on 100 000+ roms. I have my roms in folders like

America > Gamename.zip Asia > Gamename.zip Europe > Gamename.zip Prototype > Gamename.zip Unlicensed > Gamename.zip Hacks > Gamename.zip.

I don't know how the software works but it could also check the name of the directory the rom is in. Or like I said a manual mode to choose between rom suggestions when the software hesitate.

Thanks a lot and please reply to the topic!

Universal-Rom-Tools commented 6 years ago

Hi Guys, sorry for the very longtime without news...

I have a very hard time at my real work... But I'm back in 2018 ;)

with unfortunatly a bad news.

We don't add manually new roms name or CRC in ScreenScraper anymore. The "automatique" process is far enough good to add them automatically. And the number of daily request is important enough (more than 4.000.000 a day)

How it work exactly :

When you scrape a rom, it send the multiple information to the API (Filename, CRC, MD5, SHA1, filesize, system,....) If it match : great !!! If it don't match : ScreenScraper store these informations.

If these informations are send to the DB more than 5 times from different @IP. So it's considered as a "real game" so it add the information to the "Rom to associate" table. After that it check if the rom name can be associated to other rom name of the same system (or game name and lot's of stuff ^^). If it found a perfect match. ScreenScraper auto associate it to the good game and it's done. If it found an average match. ScreenScraper send the information to the validation tab for moderator ;) If it found no Match. ScreenScraper do nothing ^^ and the rom still in the "to be associated" stats.

Sorry for all the stuff you send me from a long time. Hope since, the "auto process" have done his job ;)

So I close this topic :(

PhlynnStarwind commented 6 years ago

Well that definitely makes sense. Good to know. Hope 2018 is a good year for you none the less. Thanks for the update.

On Tue, Jan 2, 2018 at 7:21 AM, Universal ROM Tools < notifications@github.com> wrote:

Closed #159 https://github.com/Universal-Rom-Tools/Universal-XML-Scraper/issues/159.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Universal-Rom-Tools/Universal-XML-Scraper/issues/159#event-1406155924, or mute the thread https://github.com/notifications/unsubscribe-auth/AfRIASMiexo_vrfOr9mJLbXOWKIHsCKEks5tGh9fgaJpZM4L3Jpa .

ghost commented 6 years ago

Thanks for your reply. Anyway I found out that it was always a matter of naming the rom correctly. Playing with the names. Removing a "." in the name helps. Sometime (rarely), I had to re-download a new rom for a particular game. Now I'm 100% scraped.