search

meleu / hascheevos

A way to check if your ROM is OK for RetroAchievements.
49 stars 5 forks source link

New RA Hashes replacing MD5 sums #19

Open Dimensional opened 3 years ago

Dimensional commented 3 years ago

It appears that RA recently changed how they get checksums for some files and systems. Now depending on the system, they get a checksum based on portions of a file, or even the contents of a rom, such as a header or executable.

The following link shows some of the changes they've done, and at the bottom of the FAQ page, they have a changelog showing they are changing from straight MD5 checksums to RA checksums that read files differently.

https://docs.retroachievements.org/Game-Identification/#playstation https://docs.retroachievements.org/FAQ/#changelog

meleu commented 3 years ago

Indeed. I should use the RAHasher.

Just need to find a way to put this as a priority in my To Do list

Dimensional commented 3 years ago

May also want to look at rcheevos. This appears to be a standalone library meant for 3rd party software. Only it's difficult to determine if it supports chd as a replacement for general cue/bin images. One thing I read in this library was supposedly the ability to add in extra callers so that 3rd party software can handle the reading of custom formats.

Dimensional commented 3 years ago

If native reading of chd doesn't already exist, may want to look into this library as well. libchdr

RandomNinjaAtk commented 2 years ago

I'm interested in seeing this project updated. I'm working on building this into a docker container for easier deployment and usage.

I want to use this as a way to clean up/validate a rom library.

meleu commented 2 years ago

wow! then maybe I should convert my code to a POSIX style so it can fit in a single alpine image!

I'm kidding... 😅

give me a week or two so I can revisit this code here.

@RandomNinjaAtk is your project on github? maybe I can help.

RandomNinjaAtk commented 2 years ago

wow! then maybe I should convert my code to a POSIX style so it can fit in a single alpine image!

I'm kidding... 😅

give me a week or two so I can revisit this code here.

@RandomNinjaAtk is your project on github? maybe I can help.

Github: https://github.com/RandomNinjaAtk/docker-hascheevos Dockerhub: https://hub.docker.com/r/randomninjaatk/hascheevos

The docker is already built and functions. However, there is no automation yet. The app is downloaded, compiled and installed and I can run it via CLI manually.

So I have much more work to do. But this is my goal:

  1. Create a Clean Rom Library for use with EmulationStation (retropie/steamdeck/etc)
  2. Ability to scrape roms for Emulation Station Platform
  3. Ability to remove dupes would be nice...

So I've done the second one in the past, but this app you have created should help me filter the roms, so hopefully reduce a collection to valid roms and then pull artwork.

I found this project from some google searching trying to find a better way to clean my library and this seems like a perfect fit.

I imagine, users will just need to dump roms into an import folder, and the docker will auto-process them via this application and scripts to scrape, organize and build the library.

Which can then be used for Batocera, Recalbox, RetroPie, SteamDeck and etc...

At least thats my goal/plan...

If you want to chat more, you can find me on discord, I have a server for my containers, link can be found on this repo: https://github.com/RandomNinjaAtk/docker-hascheevos

RandomNinjaAtk commented 2 years ago

Quick update, I updated the github repo for the docker. I also added the automation piece. I have not tested it yet, but it should work....

It also has the link for discord

RandomNinjaAtk commented 2 years ago

Tested it, and it appears to be working!

RandomNinjaAtk commented 2 years ago

I've made some good progress on it. You should give it a try! @meleu

But through some testing, I do feel like the new RA Hash verification method is greatly needed...

RandomNinjaAtk commented 2 years ago

FYI, I figured out how to use RAHash and developed my own script to process ROM files that does not use this app. It's still in it's early stages but it resulted in a lot more matches.

meleu commented 2 years ago

That's good that you're getting things done on your own!

Regarding hascheevos, I still plan to revisit my code here. But for now I'm working on the RALibretro/RAHasher side. I want to release the binaries so we can simply download it (not worrying about getting all the dependencies and compiling it locally).

My main motivation is exactly what you brought: docker containers.

Container images are supposed to be as small as possible. We shouldn't be installing all the dependencies just to build a binary and then needlessly keep them on the image.

RandomNinjaAtk commented 2 years ago

It would be nice to have a pre compiled Linux binary, windows binaries are already provided.

I hope you give my container a spin and see what you think.

I've always felt managing ROMs and having an accurate library without dupes is a real pain to manage and I've not found a good way to do that until I stumbled across this project.

I just want to take it much further and I think so far the results have been good...

I also posted on the RA forums in a couple of spots. I wish RA had a better API for gathering info.