N8 Enhancement/Suggestion

[maniac79]

I'm suggesting to re-organize the licensed NES ROMs in the N8 pack. As you probably know, Nintendo had 2 different PAL regions back in the days, PAL A and PAL B, both with their unique set of games. I suggest following structure:

1 US - A-F 1 US - G-Q 1 US - R-Z 2 Europe |- 1 PAL A - A-Z [296 games released + diff. language versions] (1 would be missing in the folder cause PAL A ROM undumped according to No-Intro & Bootgod: Miracle Piano) |- 1 PAL B - A-Z [340 games + diff. language versions] |- 2 PAL A not in PAL B - A-Z [8 games] |- 2 PAL B not in PAL A - A-Z [53 games]

I have already prepared the Europe part for myself like this and could submit a PR soon-ish. Before submittag the PR I would check for games which [I feel] don't really belong where they are today. For example, I think 25th Anniversary Super Mario Bros. (Europe) (Promo, Virtual Console) belongs into the Revisions folder as this is a later re-release of the game and was not originally released as part of the set back in the days. Currently it's only found in the folder 2 Europe - A-Z/PAL A-M/ and nowhere else. For games available in different languages, I would place all available in the 'base folders' (1 USA/2 Europe). My goal is to have clean sets with retro ROMs as they were released back in the day for the different regions without cluttering them too much. I personally think having all the different languages in the 'base folders' is too much but as I'm German I'd favor these to which I'm sure no one else here would agree - hence proposing to put them all there for convenience.

I would need some time to check everything to ensure nothing is lost and some more time to prepare a clean US set in the same way.

Opinions?

[Also, is there any other way to discuss stuff like this here on GitHub without opening an issue?]

[frederic-mahe]

@SmokeMonsterPacks what do you think?

As you probably know, Nintendo had 2 different PAL regions back in the days, PAL A and PAL B, both with their unique set of games.

Playing the devil's advocate here. PAL A and PAL B were covering different geographical regions (A: GBR, UKV, ITA, AUS; B: FRA, NOE, ECC, FRG, ESP, SCN, SWE), if you split them then someone looking for a specific game must know in advance if it is a PAL A or PAL B.

Why not adding the information to the ROM name? Elite (Europe) (En,Fr,De).nes would become Elite (Europe) (PAL B) (En,Fr,De).nes and Elite (Europe) (PAL A) (En,It).nes (assuming both exist and are different). The games would be listed side by side, it would make clear that PAL A and PAL B exit and are different.

For example, I think 25th Anniversary Super Mario Bros. (Europe) (Promo, Virtual Console) belongs into the Revisions folder as this is a later re-release of the game and was not originally released as part of the set back in the days.

You are correct.

[SmokeMonsterPacks]

If you have an easy way to do that, I say go for it :)

Normally, the latest revision of a game goes in the official folder, and older revisions go in Revisions. I know no one wants to put the original SMB in Revisions, but that is just the logic there.

[maniac79]

I wish I knew an easy way to do that. It's a long and blocky road but I'm up to the task and I'd say I'm already 50% done with reorganizing everything (I'm working with the latest No-Intro set, which means we'll have correct and uptodate hash information and file names). :)

I agree with you, @frederic-mahe, we should not assume people know which games came out in which PAL region. Hence I propose to put ALL official PAL games into the official folders [latest retro version]. But for people who know where to look and who would appreciate a clean look at the games released in each of the regions, I will create subfolders in 4 Game Series Collection/. OK?

Why not adding the information to the ROM name? Elite (Europe) (En,Fr,De).nes would become Elite (Europe) (PAL B) (En,Fr,De).nes and Elite (Europe) (PAL A) (En,It).nes (assuming both exist and are different). The games would be listed side by side, it would make clear that PAL A and PAL B exit and are different.

I'd rather go by the official No-Intro naming conventions and not add additional information.

Normally, the latest revision of a game goes in the official folder, and older revisions go in Revisions. I know no one wants to put the original SMB in Revisions, but that is just the logic there.

@SmokeMonsterPacks: I agree with the logic of not having the first SMB version in the official folder very much but I'd also rather not have later re-releases there, e.g. 'Virtual Console', 'Gamecube', ... in the official folders - I'd rather place them in the Revisions folder too. Agree?

[frederic-mahe]

Good compromise, thanks @maniac79 !

[maniac79]

@frederic-mahe If it's not too much to ask for, it would be great if work on the N8 pack can be frozen while I work on a huge update.

My work, unless stated otherwise, is based on No-Intro's Nintendo Entertainment System (20200329) [can be found on archive.org]. I'm rebuilding most of the N8 pack folders manually, ROM by ROM. There will be some folders which I won't touch (e.g. FDS stuff, I'll include all details in the PR).

Already complete

1. Rebuilt all main folders [US, Europe, Japan, Other Regions, Unlicensed] *)
2. Rebuilt Prototype/Samples/Revisions folders (adding ROMs absent in No-Intro from previous pack version)
3. Rebuilt Tools folder (adding ROMs absent in No-Intro from previous pack version)

*) The US and Europe folders can then be seen as "static" or "final" unless someone unearths a previously unknown retail/licensed game. These folders contain exactly the 677 licensed US retail releases and 348 licensed EU retail releases [English versions]. I'm quite proud of this part of the update. :)

Open

1. Rebuild 4 Game Series Collections folder based on the No-Intro specs - I assumed this would be the most complex task and I was right but I still underestimated it by a huge margin - I think I'm ~60% done and it took me probably ~10 hrs so far. My vacation ends tomorrow, so it could take maybe 1-2 weeks to complete the remaining folders. **)
2. Build new SMDB file and ensure nothing is missing from the previous version.
3. Ensure the new pack is in line with the latest No-Intro set. ***)
4. Test SMDB file.
5. Submit PR.

**) Note that this part of the update also includes a major cleanup - a few examples:

1. There were inconsistencies in the "Chronological" folders which I noticed when I rebuilt the "By month" folders by using the "Yearly folders" as the base [fixed].
2. There are many inconsistent file names, hashes, etc spread among many Game Series Collections subfolders. My update should fix all of these issues.

***) I'm well aware that there's a new No-Intro set [20200405-041607] and before I submit my PR, I'll make sure the pack is in line with it.

OK?

[frederic-mahe]

If it's not too much to ask for, it would be great if work on the N8 pack can be frozen while I work on a huge update.

@tjanas and @SmokeMonsterPacks please comment if you have pending modifications. The main issue is that tjstogy also started to work on an update of the Game Series Collections.

Dennis, is there a way you two could work together on that? if tjstogy is willing of course.

[maniac79]

Is tjstogy working on the N8 pack too? 🤔 There were a few wrong revisions and I believe a lot of „wrong“ (if you consider different from No-Intro as wrong) hashes in the main folders. If the Game Series updates are based on the current main folders, then I suggest to let me finish first. I have put ~10 hours into the Game Series folders alone up to now and another 6-8 hours probably into the main and betas/prototypes/... folders.

[tjanas]

This is great. Please let me know if I can be of any help. Over the past few months, many Spanish translations were added, and I noticed many duplicates hashes and unnecessary additions (like translations of GoodNES bad dumps and overdumps).

[tjanas]

@maniac79 how are you comparing hashes vs. nointro, since the latter hashes the unplayable headerless rom?

[maniac79]

@tjanas I am leaving the majority of Hacks/Translations untouched, if you could check for an issues you mentioned later (after I‘m done with my current tasks), it‘d be great.

clrmamepro seems to ignore the headers, I was able to verify the Headered ROM set I‘m using with a DAT file downloaded from datomatic.

[tjanas]

@maniac79 i have a rebuilt Game Series Collections from tjstogy himself, the creator of the collections. I was thinking about creating a branch in my fork with his changes, and perhaps you could compare with the ones you have been working on?

[maniac79]

If tjstogy worked with the hash infos from the base folders as they are in the SMDB pack file today or left most of the information from the same pack file untouched, then my guess is that he has built it on outdated CRC hashes and older revisions of a some games. If he has created the Game Series Collections folders from scratch with the latest No-Intro ROM specs, then I spent a lot of time in vain but at least we would have an awesome pack when our work is merged. 😄

But then someone else has to go over the Game Series Collections and check if they use the best revisions of each ROMs from the base folders and check if they all match the CRC hashes. I have spent another 2 hours today on the Game Series Collections. Meaning I spent 12+ hours manually copying 3.000+ ROMs into their respective folders to make sure the correct No-Intro ROM hashes are used for each ROM and had about ~1.500 left. Plus some additional clean ups (e.g. some of the Top 10s have totally changed in time and I have updated them, one of the Top 100 was a missing a game which I added, ...). Don‘t get me wrong, I‘m not sour or anything, it‘s all cool and things like this can happen I guess, just don‘t feel to start the Game Series Collections over from scratch and would still want to contribute everything else at least.

Btw, I am pretty sure that there‘s the same issue with the NES & Famicom Mini ROM sets as there was with the Genesis Mini ROM set. What I mean is: It seems someone has put the games together which are on the NES and Famicom Mini but they are not the dumps from the machine itself, which could potentially be different from the ROMs which were used in the Game Series Collections. This is an update I could check in detail and add after tjstogy‘s and my works are merged.

Maybe I should stop and wrap up my activities ASAP and share all my updates except the Game Series Collections? What do you think @frederic-mahe?

[frederic-mahe]

@maniac79 don't worry, your work won't be lost.

Game Series Collections can be seen as different viewpoints of what's in the main folders (US, J, EU). That's why I think work on the main folders has priority over work on Game Series Collections, to avoid desync. Since you improved and fixed both Main and Game Series, I suggest you submit a pull request with all your changes (main folders and Game Series).

Once merged in the master branch, I will assess the differences with what tjstogy has, and what needs to be ported into the master branch. The most difficult cases we'll be discussed publicly in a github issue. Maybe @tjanas will join us?

Regarding, the NES & Famicom Mini ROM sets, please open a dedicated issue if you think these sets need to be fully reviewed, once your other changes are merged.

[tjanas]

I would be happy to review these changes, as well as the ones by tjstogy.

[tjanas]

Here are the proposed changes to Game Series Collections by tjstogy.