search

andresdelcampo / GameList_Editor

This repository intends to contribute to GameList Editor development and bring it back to life with the latest systems, etc.
GNU General Public License v3.0
33 stars 0 forks source link

Scraping scrapes wrong systems... #13

Closed Kugelblitz360 closed 8 months ago

Kugelblitz360 commented 8 months ago

So I briefly tried out updating some games, but ran into the problem that the scraper picked the wrong system. I got Atari ST screens and art for some C64 games for example. Do I need to set extra parameters? My Screenscraper credentials are correct.

andresdelcampo commented 8 months ago

Hmm... have you checked if the image was wrong in the database in https://www.screenscraper.fr/? I have seen those cases of wrong images uploaded to the database before. If you can confirm one example where it does not match the database, I will investigate in the source code.

Kugelblitz360 commented 8 months ago

Two examples here. First trying an Amstrad CPC game returns the media for the Amiga version. Then trying a C64 game returns the media for the Sega Master System version. Clipboard Image Clipboard Image (1) Clipboard Image (2) Clipboard Image (3)

andresdelcampo commented 8 months ago

The first game did not seem to be in the DB, but the second is, and you are right, it does not match. https://www.screenscraper.fr/gameinfos.php?gameid=36304&action=onglet&zone=gameinfosmedias

The query itself in code seems right -it only passes ROM CRC and system ID, but maybe the systems were mismatched or maybe altered at the server... no, those still are 65 and 66 for CPC and C64... I will need to investigate a bit deeper to see what is going on. The one for CPC I could understand -it did not have it in that system and it tried a different one- but the one for C64 should have worked? I will try to get those roms -to get their CRC- and try the service API to see where the issue is -in the server or the app. Give me a few days -shouldn't take long.

andresdelcampo commented 8 months ago

Weird, I wonder why it still works for me. I will try alternative roms... (so far I do not repro neither the date issue nor the scraper one, but I will keep trying over the weekend as you have definitely proven it possible). image

Kugelblitz360 commented 8 months ago

My only good guess would then be locale settings in Windows - my system is set to German and that might mess up date formats in the app?

andresdelcampo commented 8 months ago

Ok, on the scraper part... no luck on another 007 CPC alternative rom -different hash.

But I managed to repro the C64 one... !

I can confirm that GameListEditor is passing the right system, and the CRC -I see no issues in the API call. Unfortunately the ScreenScraper site does not return the right info as the ROM CRC is not matching the Commodore 64 one... https://www.screenscraper.fr/rominfos.php?romid=874522 84020FA5 but 86996661 instead (52F76B5 in hex). So given that it figures out that the game is the right one, I suspect someone entered the right CRC (for C64) in the wrong system (Sega) in ScreenScraper...

Actually, try this: https://www.screenscraper.fr/rominfos.php?romid=874522 -notice that the page says Commodore 64. Now click in Summer Games (Game Name field).... notice where you land. https://www.screenscraper.fr/gameinfos.php?gameid=960&action=onglet&zone=gameinfosroms -see which system it is... this is something to report to ScreenScraper.fr. I am afraid I cannot do much about it from the GameListEditor point of view, and I have no direct contact with the site guys. I would suggest you take it with them if interested in fixing it... I am afraid it will be case by case individually as this is strictly about the site contents.

Kugelblitz360 commented 8 months ago

Ah, if you search by CRC you of course will run into issues on computer software, especially disk based or cartridges with integrated save, because the CRCs do change and there are MANY versions anyway. They are only useful on consoles really, the CRC lists for Home Computers are very incomplete. For these cases the scraper in Emulation Station allows a search by name (and manual input of the search string) and then shows a list of possible entries. If that is out of scope for your project, I understand and just close the request, but you might want to add a remark that scraping is by CRC only. Thank you!

andresdelcampo commented 8 months ago

It could be if it worked, but the funny part is that ScreenScraper had the system already as it was passed as part of the search. It knew it was a C64 ROM, not a Sega one, and it correctly identified it was Summer Games. It just made the leap to Sega for some strange reason... While changing the scraper functionality is not in scope at the moment, I will keep it in mind if I get to expand the Scraper. It is not so much "my project", I kind of adopted it quite mature, as I liked it and I wanted to expand it a bit with video and so on and it was sadly abandoned, but Scraper to me was rather secondary -particularly massively scraping- as it is waaay too slow and there are better alternatives with other software. But when it comes to the UI and browsing the game collection and editing the gamelist, very few (none?) can beat GameListEditor :).

Kugelblitz360 commented 8 months ago

You are right, as an Editor it is #1, followed by Batgui.