libretro-thumbnails / libretro-thumbnails

Thumbnails for RetroArch
https://github.com/libretro-thumbnails/libretro-thumbnails#libretro-thumbnails
538 stars 74 forks source link

ePSXe covers #31

Open 1oglop1 opened 6 years ago

1oglop1 commented 6 years ago

Hi, I'm new to RetroArch and it seems that thumbnail functionality is based on the game title and only then the thumbnail is loaded. Please correct me if I'm wrong.

The problem of this approach is that names may not always be same/similar, but maybe there is a different approach.

Anyway, here is what I observed using ePSXe - it has really simple functionality for game library.

  1. Open Gamelist
  2. Set your game folders
  3. Hit Refresh
  4. Click Get covers image

Program loads game info and then it downloads the cover (dunno where from or how yet), and stores this file in Covers folder as serial number. I think this might be better idea to match games with covers correctly. image

In the end there could have been some functionality to locate and download thumbnail from this repo into retroarch based on content available.

Let me know what you think.

RobLoach commented 6 years ago

Agreed, matching against the serial instead of the name would be great. Unfortunately, RetroArch in its current form does not do that. It matches the menu name against thumbnail filename.

Does Gamelist allow renaming and updating the filenames to match that? It is quite the manual process currently. There is a check that goes on over at: https://github.com/robloach/libretro-thumbnails-check

1oglop1 commented 6 years ago

Once Covers has been downloaded and stored in ePSXe/covers program just checks that folder and you can rename the file to match different Serial number. So ePSXe seems to work only with Serials. - we could create a fall back function to use file name if serial is not found or so.

I just found out at the size of thumbnail lib is huge and there are several duplicates, might be a good idea to come up with some naming in case game cover is same for multiple serial numbers and name same pictures like: SCES-1234 SCES-5768 or create database of these duplicates Because there are multiple copies of same image for example: https://github.com/libretro-thumbnails/Sony_-_PlayStation/blob/master/Named_Boxarts/40%20Winks%20(Italy).png and https://github.com/libretro-thumbnails/Sony_-_PlayStation/blob/master/Named_Boxarts/40%20Winks%20(Europe).png

Are exactly same files.

On the top of that, this repo seems to be using git submodules which is not great approach to manage multiple repositories as it requires quite some work to manage. And one needs to update submodules every time child repo is updated. And manual checking could also be quite time consuming. (I found out that 40 winks has 3 images in Sony Playstation repo, but I cloned this one and got only 2) So there is definitely space for improvement by using CI/CD like Gitlab CI or travis. Or using git subtree.

My personal preference is Gitlab CI as it is very easy to setup runner on local PC which would eliminate need to download entire repository every time to do the process.

So far I did not dive much into a process how everything is managed - so I'd need some introduction to implement it properly.

RobLoach commented 6 years ago

Git subtree is a great idea. Do child repositories have the git history of the parent? We still want an easy way to manage the individual repos so that when you check things out, you don't need 30GB to check it out.

Had considered using symlinks for the duplicate files, but unsure how/if that works with the archives or all platforms.

Would love to see more adoption of gitlab ci.

1oglop1 commented 6 years ago

Okay, gimme some time to prepare examle workflow and once that done I'll update this issue.

1oglop1 commented 6 years ago

@RobLoach My second thought was to find good alternative to the covers if there is a need to download them to the local computer, because storing binary data in git repository is not really optimal solution since all changes are still stored in .git directory which is roughly same size as the content but has no value for end user.

Update to ePSXe, I sniffed packets from the program during the cover download process and found out, that it goes to http://epsxe.com/files/pccovers/0/8/081acb69cbcea2c7ff88e2559a1436ed.bmp The url above is accessible from http://epsxe.com/files/pccovers/, rest of the URL is some random database entry where hash helps to speedup the search.

I just found out about large database here, they might be willing to cooperate, so we might be able to change whole git database into database of URLs and use API automate to process of downloading of thumbnails. https://psxdatacenter.com/pal_list.html

RobLoach commented 6 years ago

PSXDataCenter is scraped for libretro-database over at: https://github.com/robloach/libretro-database-psxdatacenter

It scrapes data for Sony - PlayStation Portable, Sony - PlayStation, and Sony - PlayStation 2. The serial data is available in libretro-database.