search

mamedev / mame

MAME
https://www.mamedev.org/
Other
7.95k stars 1.98k forks source link

mame 0.226 fails to verify some files despite checksums being correct (Linux) #7500

Open worthlesscog opened 3 years ago

worthlesscog commented 3 years ago

Mame 0.226 seems to fail to verify some roms despite checksums being correct.

Ubuntu 18.04.5 LTS, Mame 0.226 from Cesare Falco's PPA

/mame$ file /usr/games/mame 
/usr/games/mame: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=bce29a15d8c7ba35b6d2e13c3ea8ab7c3a61f852, stripped
/mame$ mame -version
0.226 (unknown)

Source -

/mame$ mame -listroms hapyfsh2
ROMs required for driver "hapyfsh2".
Name                                   Size Checksum
flash.u6                         2214592512 CRC(3aa364a2) SHA1(fe09f549a937ecaf8f7a859522a6635e272fe714)
flash.u28                        2214592512 CRC(f00a25cd) SHA1(9c33f8e26b84cea957d9c37fb83a686b948c6834)

/mame$ unzip -v hapyfsh2.zip 
Archive:  hapyfsh2.zip
 Length   Method    Size  Cmpr    Date    Time   CRC-32   Name
--------  ------  ------- ---- ---------- ----- --------  ----
2214592512  Defl:N 2024656198   9% 2020-11-06 16:06 f00a25cd  flash.u28
2214592512  Defl:N 1307980277  41% 2020-11-06 16:06 3aa364a2  flash.u6
--------          -------  ---                            -------
4429185024         3332636475  25%                            2 files

Storing hapyfsh2.zip in roms makes verify pass :

/mame$ mame -verifyroms hapyfsh2
romset hapyfsh2 is good
1 romsets found, 1 were OK.

This returns instantly - there isn't time to read the whole file so I assume it's just checking the CRC stored in the archive.

Extracting archive to create hapyfsh2 makes verify fail :

/mame/roms/hapyfsh2$ ls -la
total 4326360
drwxrwxr-x     2 games games       4096 Nov 20 13:58 .
drwxrwxr-x 37854 games games     991232 Nov 20 13:58 ..
-rw-rw-r--     1 games games 2214592512 Nov  6 16:06 flash.u28
-rw-rw-r--     1 games games 2214592512 Nov  6 16:06 flash.u6

/mame/roms/hapyfsh2$ sha1sum *
9c33f8e26b84cea957d9c37fb83a686b948c6834  flash.u28
fe09f549a937ecaf8f7a859522a6635e272fe714  flash.u6

/mame$ mame -verifyroms hapyfsh2
hapyfsh2    : flash.u6 (2214592512 bytes) - INCORRECT CHECKSUM:
EXPECTED: CRC(3aa364a2) SHA1(fe09f549a937ecaf8f7a859522a6635e272fe714)
   FOUND: 
hapyfsh2    : flash.u28 (2214592512 bytes) - INCORRECT CHECKSUM:
EXPECTED: CRC(f00a25cd) SHA1(9c33f8e26b84cea957d9c37fb83a686b948c6834)
   FOUND: 
romset hapyfsh2 is bad
1 romsets found, 0 were OK.

This check takes a long time so it must be reading the files but failing to calculate the checksum.

ghost commented 3 years ago

I'd hazard a guess you're running out of RAM decompressing them, this is an issue on 32-bit at least, there's no real workaround, the ROMs are bigger than the maximum amount of RAM you can allocate.

worthlesscog commented 3 years ago

This box has 32GB physical, 64 bit O/S and 64 bit mame according to file. I guess it could be process limits but I thought they were set to unlimited on this box. I will check. Flagging the data as bad because mame can't allocate enough memory to checksum is a bit poor. I'm not a C++ programmer but surely you get a warning if you try to allocate too much?

worthlesscog commented 3 years ago

Interestingly Mame64 on Windows 10 will verify both 7z archives and extracted files for hapyfsh2 whereas the Linux version will only verify the archive version as correct. I thought these were the process limits, but I'm still trying to work out if there's another doohickey somewhere that's actually in charge of process limits -

/mame$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 127353
max locked memory       (kbytes, -l) 65536
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 127353
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

Neither runs correctly but this set is listed as preliminary I think so that's probably why. Running the Linux version with an archive doesn't complain that the set is broken or incorrect.