Universal-Rom-Tools / Universal-XML-Scraper

Scraper de Rom
195 stars 46 forks source link

Missing File #159

Closed Universal-Rom-Tools closed 6 years ago

Universal-Rom-Tools commented 7 years ago

If you want to add some new Rom to the screenscraper DB,

just read this : https://github.com/Universal-Rom-Tools/Universal-XML-Scraper/wiki/Rom-Missing-on-Screenscraper

pillemats commented 7 years ago

ok i will do that ! but here is another missing file from dreamcast with crc

please add it to the db !

Thanks _Dreamcast_missing.txt

Universal-Rom-Tools commented 7 years ago

still no CRC @pillemats

(check here : https://github.com/Universal-Rom-Tools/Universal-XML-Scraper/wiki/Rom-Missing-on-Screenscraper)

pillemats commented 7 years ago

i used that how to before....

in the morning i have deleted all files and begin a completeley new search with uxs CRC+Filename and here are the results. i hope you can use ist now. if not, i not know what should i do any more. i hope thats all what is missing :-)

_Dreamcast_missing.txt _Game Boy Color_missing.txt _Game Boy_missing.txt _Playstation_missing.txt _PSP_missing.txt _NES_missing.txt

bibi32 commented 7 years ago

_Game Gear_missing.txt _Game Boy Advance_missing.txt _Game Boy Color_missing.txt _Game Boy_missing.txt _Megadrive_missing.txt _Super Nintendo_missing.txt

Universal-Rom-Tools commented 7 years ago

@pillemats DC : Still no CRC..................... Just download F-CRC Calculator, put your is in it, be sur MD5 AND CRC are check and generate the file.... GBC : Done GB : Done (3 of them already exist in DB) PSX : Still no CRC..................... PSP : Still no CRC..................... NES : Already in DB Please Pillemats, I loose a lot's of time checking these file and importing them... It's your third attempt... please take care and give me "good" file.

@bibi32 GG : Done GBA : Done GBC : Already exist in DB (certainly as Orpheans) GB : Already exist in DB (certainly as Orpheans) MD : Done SNES : Done

pillemats commented 7 years ago

Thank you ! I will use the crc calculator to create a"good" file fit you :-)

pillemats commented 7 years ago

so... i tryed the f-crc calculator an now it create a txt file like this

Aliens vs. Predator - Requiem (USA).iso 678EC8F8 396 490 752 01:20.23 2007-11-15 7C2C0459328199CC2A6BAA7F483892D1

it is an psp file ! is it good enough to add this to db ?

EtherBoo commented 7 years ago

Famicom Disk System Missing _Family Computer Disk System_missing.txt

pillemats commented 7 years ago

psp.txt

Universal-Rom-Tools commented 7 years ago

@EtherBoo FDS : Done

@pillemats YES !! You did it ;) It's done ;) now you have to associate the orphean rom to there respective game (or ask to create the game if missing ;) )

EtherBoo commented 7 years ago

@Universal-Rom-Tools Tried rescraping FDS, still got a ton of missing ones. Also some missing from 7800, Virtual Boy (two ROMs don't appear to be in your database though), and WonderSwan _Atari 7800_missing.txt _WonderSwan_missing.txt _Virtual Boy_missing.txt _Family Computer Disk System_missing.txt

pillemats commented 7 years ago

dc.txt

Universal-Rom-Tools commented 7 years ago

@EtherBoo A7800 : Done WS : All are already in DB VB : Done FDS : Done (with a lot of already exist)

FYI : When I said "done" it mean I add the "roms" to the DB. But, even if majority are "autoassociated", a lot's of rom aren't associated to a game. So you need to do it and look for your "orpheans" roms to associate them.

@pillemats DC : Done ;)

EtherBoo commented 7 years ago

@Universal-Rom-Tools Thank you. Maybe you can help me with a clarification.

I'm still getting missing files when I scrape. How do I associate the orphan ROMs? Am I wrong in thinking that I shouldn't be missing anything after a scrape?

paradadf commented 7 years ago

@EtherBoo go into www.screenscraper.fr and on the top you have a "tab" called roms. Select your system and you'll see a list of all available roms for that system on the database. Some will be associated to a game, some won't. And that is what you need to do. When the admins import new roms, the is an autoassociation process, but not every rom gets a match, thus contributors beed to do it manually. Of course, all this assumes the game already exists. If that is not the case, you can siggest to add a new game entry to the database. And I say entry, because having a game listed and having its data and media are different things. So even if a game exists, and your rom is associated to that game, if it lacks all data, it won't get scraped.

EtherBoo commented 7 years ago

@paradadf Ah. OK, I'll work on that tonight. Thanks a bunch!

pillemats commented 7 years ago

psx.txt

paradadf commented 7 years ago

@Universal-Rom-Tools be awere of those lists full of crap like txts and xml files included. Also, I noticed you get the full relative path instead of the basename (subfolder/filename), so you might wanna check that.

pillemats commented 7 years ago

@Universal-Rom-Tools

how should i associate the orphean rom to there respective game? i tink i must do that on screenscraper.fr ??? or is it enough to upload the new missing file here and you can import it automaticly ?

Universal-Rom-Tools commented 7 years ago

@paradadf don't worry ;) I clean and check before importing ;)

@pillemats I can't import you psx file... The name are "wrong".... All your file seems to be "USA"... But after some check some are EU or German..

Exemple : Resident Evil - Survivor (USA).bin is a EU rom Resident Evil 3 - Nemesis (USA).bin is a German one...

If I import them, it will add crape to the DB, tagging German Rom as USA :S and it's not good...

And like @paradadf explain only 2 messages up, you need to associate the orphans manually on www.screenscraper.fr

pillemats commented 7 years ago

oh this is not good.... is it possible to add only the filename from the file ?

so i can scape by filename only

Universal-Rom-Tools commented 7 years ago

@pillemats nop.. it's impossible...

Juste take a "good" romset, or try to scrape in local with CRC, or try the Experimental function to Hash with SSH, or rename your rom (you did it once) to match the one on ScreenScraper... (but it's not the best idea)

EtherBoo commented 7 years ago

Working on NES now. Downloaded a No-Intro set, a bunch of them did not get matches _NES_missing.txt

Thanks for all the work you guys do. You're the best! .

bibi32 commented 7 years ago

_Game Boy Advance_missing.txt _Mame_missing (neogeo).txt _Megadrive_missing.txt

hollowfaust commented 7 years ago

Sources TOSEC et GOOD : _CPC_missing.txt

Roms testées et fonctionnelles : _Playstation_missing.txt _Mega-CD_missing.txt _PC Engine CD-Rom_missing.txt

Universal-Rom-Tools commented 7 years ago

@EtherBoo Sorry, it's not a "good" no-intro romset. naming convention aren't respected :( I can't import that (no info about region for exemple or hack/unl/beta/proto/...)

@bibi32 GBA : Done Neogeo : Done

@hollowfaust CPC : Done PSX : Sorry, rom aren't well named (not comming from a good romset) Mega CD : Sorry, rom are well nammed ;) but CRC and MD5 missing... Please use F-CRC.

hollowfaust commented 7 years ago

New set of PSX missing roms: _Playstation_missing.txt

Thanks for your work !

bibi32 commented 7 years ago

_Game Boy Advance_missing.txt _Mame_missing(neogeo).txt _Super Nintendo_missing.txt

Universal-Rom-Tools commented 7 years ago

@hollowfaust Don't know what happen on your Hash, but every MD5 are ate '-1' so I can't import it :(

@bibi32 GBA : Done NeoGeo : Already added (thank you for your precision Neogeo / not real Mame rom ^^) SNES : Done

hollowfaust commented 7 years ago

@Universal-Rom-Tools Right, I think its because I used the SSH scraping method. Here is the file again. _Playstation_missing.txt

Universal-Rom-Tools commented 7 years ago

@hollowfaust Arg... still not good, this time there is MD5, but no CRC :( (and it's "normal" with the experimental SSH hash because it only hash in MD5. It's normal too in the standard mode, file bigger than 500Mo are only CRC hashed).

The best way to generate a "good"file I can use :

Sorry for the inconveniance.

hollowfaust commented 7 years ago

ok tell me this time I have everything :) _Playstation_missing.txt

EtherBoo commented 7 years ago

Going through more systems now. I'm finding a lot of games that are incorrectly associated - so they pull back incorrect metadata.

Here's two recents scrapes.
_PC Engine_missing.txt _Master System_missing.txt

paradadf commented 7 years ago

@etherboo please, don't forget to make suggestions on the database to correct those. Thanks in advance!

EtherBoo commented 7 years ago

@paradadf

I've made suggestions in the comments of those games - not sure if that's the right way to do it. The issue is the ROM is associated to the incorrect game. One example is Doraemon. It's associated to another game called Doraemon - Giga Zombie no Gyakushuu, so even though both games exist in the database and have ROMs associated with them, UXL seems to pick Doraemon - Giga Zombie no Gyakushuu over just plain Doraemon. I've found a few games like this.

pillemats commented 7 years ago

_Saturn_missing.txt

bibi32 commented 7 years ago

_Game Boy Advance_missing.txt _Nintendo 64_missing.txt

paradadf commented 7 years ago

@etherboo commenting is enough, thanks!

Universal-Rom-Tools commented 7 years ago

@hollowfaust PSX : Done

@EtherBoo PC_Engine : Your file are in "cue". Are thez PC Engine CD ? MS : Done

@pillemats Jaguar : CRC Missing (Please use F-CRC)

@bibi32 GBA : Done N64 : Already Exist in DB

EtherBoo commented 7 years ago

@Universal-Rom-Tools Thank you and yes, the cue files are for PC Engine CD.

To anyone.... When I scrape Arcade, I am not getting wheels on the majority of games I scrape. I have the mix set up for standard2, and it works for most systems, but does not work for Arcade for some reason. Any ideas?

Universal-Rom-Tools commented 7 years ago

@EtherBoo Can you make an issue for that (I will take a look) PCE CD : done

EtherBoo commented 7 years ago

@Universal-Rom-Tools Sure. Where do I do that? On the ScreenScraper website?

Universal-Rom-Tools commented 7 years ago

Here ;) in a new issue.. It's just for me to create a reminder ;)

hollowfaust commented 7 years ago

_Playstation_missing.txt

sbowler commented 7 years ago

_Megadrive 32X_missing.txt _NES_missing.txt _Nintendo 64_missing.txt _Playstation_missing.txt _Super Nintendo_missing.txt

I've tried to go on the site and match some of these. It's kind of hard when the site doesn't seem to work with chrome and some of us don't speak French. Other browsers support for translations sucks or is not existent. I'm happy to try to help out on the site as best I can. It's kind of hard to figure out my way around though.

Universal-Rom-Tools commented 7 years ago

@hollowfaust No CRC, sorry... I can't import it.

@sbowler Hi, You can change the ScreenScraper Website language just by clicking on the little flag ;) (all tranlsattion aren't done, but it's pretty usable in you own language ;) ) Concerning your files : Megadrive 32X : Pretty weird... lot's of game seems to be only Megadrive, not 32X... Can you check your Romset ? NES : Glups... can' t import game file like : MARIO'\~1.NES or MARIO2\~1.NES + there is no "country" Tag... Can you check your romset ? N64 : A lot's imported ;) PSX : No CRC, and no Country tag :( sorry, can't import that. SNES : Imported ;)

EtherBoo commented 7 years ago

Wondering if someone can help me out or if there are changes I need to propose on the site. I'm using Mix Standard 2 Images (Screenshot and Wheel Art).

Besides Arcade not scraping Wheel Art on most arcade games (it grabbed it for Ultimate Mortal Kombat 3 oddly enough), consoles seem to be grabbing wheel art somewhat sporadically. For example, I just scraped SNES and Donkey Kong Country scraped the Screenshot and Wheel Art, but only scraped the Screenshot for DKC2 and 3.

Here's a few more missing files to add:

_Super Nintendo_missing.txt _Game Boy Advance_missing.txt _Game Gear_missing.txt _Megadrive_missing.txt _Nintendo 64_missing.txt

I may have made a huge mistake with GBA in that I stripped the numbers off the filenames because they wouldn't scrape on the last Pi i put together. Wondering if that's why I have so many this time.

sbowler commented 7 years ago

Thanks @Universal-Rom-Tools. My NES roms collection I've had since like 2000 or so for 15 years+ so it's probably a bit unique. I believe those mario games are hacked mario fan type games so that's no problem.

I thought that megadrive and sega 32x were pretty much the same thing, but maybe it makes a difference? I may have downloaded a megadrive rom set instead of a sega 32x one based on what you are saying.

I also saw some games not matched up that didn't appear in the orphan list on the site. Do you have to import them to the site before they show up? I was thinking the tool just added them automatically when it ran.

I'll have to double check, but it seemed like there were lots of games that weren't matched up and I don't think appeared in the missing list. I'll check that and let you know. Thanks so much.

patriccote commented 7 years ago

Hi,

I have a NO-INTRO romset (nes - snes - n64) but in a .7z archive. Unfortunately, Universal-XML-Scraper was not able to hash them and return the following error message "Le chemin des ROMs n'est pas accessible" (ROM path is not accessible). I renamed everything to ZIP and the hash worked for about 50% of them.

Do I need to send you the _missing.txt -or- How can I work with the .7z directly with Universal-XML-Scraper ?

(Here's the _missing.txt just in case renaming them to ZIP was a good idea =)

_NES_missing.txt

_Super Nintendo_missing.txt

Thanks !

patriccote

sbowler commented 7 years ago

So apparently I was very misinformed when it comes to sega systems. Looks like I probably had my roms in the wrong folder. Maybe this one might be better or make more sense.

_Megadrive_missing.txt

Also re-scanned n64 and had these pop up that I think might have not been in the last list. _Nintendo 64_missing.txt

Thanks