Previous iterations of the code had a singular string decode/encode routine for Gen1 & Gen2. The games have 256 different characters, and the text entry field for Japanese (or other games) were not varied between languages to have clashing characters. However, Box Names have a different keyboard entry, which has ligatures (multi-characters) that are unique across languages.
So, we needed to refactor the string logic to be language specific (and essentially Gen1 / Gen2 specific as well). Now, we have APIs that can accept language to select the correct text character encoding table. To use these APIs correctly, this required updating the Language/Version tracking of a Gen1/Gen2 save file, as this language will be used to get the right set of characters. Rather than assuming English (or detecting based on current nickname), we can do a little better by caching what language we think the save file is by looking at the file name. Granted, this doesn't work at all for Virtual Console saves (always named sav.dat), so we also need to provide a program setting to allow users to specify a forced value if none is detected from the filename.
Extending this logic to Generation 3, we can also cache the assumed language & version for these files as well. Helps eliminate the "what version is this" popup for FireRed/LeafGreen, which is needed to show the correct stats & sprite for Deoxys. Players can still manually change it with the Block Editor, and if none is detected from File Name, they can still answer the popup. Might be problematic if people randomly name their save files and hit any of the recognition patterns, but that's on them for naming things weirdly.
Since our brains are in Gen1/2 mode, we can also resolve the issue / old cruft of PokeList1/PokeList2. In these GameBoy save files, the game stores each box as a serialized list of entities, rather than just raw slot data like future games. When PKHeX first implemented Gen1/2 support, the decision was made to "unpack" these box objects into raw slot data, so that the existing APIs can read/write data without any extra abstraction. The code was implemented and basically remained unchanged for 6+ years, as it did its job.
When looking at the logic, it creates a new list object, loads all PKM from the box-list, then writes all PKM individually to a hidden buffer used for the raw slot data API. We can do a little better by just copying directly, no allocation required. Additionally, we can try to retain the original data (if uninitialized && unaltered by the user during editing), which allows for the program to be used without disturbing any set-up glitches.
Closes #4027 Closes #4224
Previous iterations of the code had a singular string decode/encode routine for Gen1 & Gen2. The games have 256 different characters, and the text entry field for Japanese (or other games) were not varied between languages to have clashing characters. However, Box Names have a different keyboard entry, which has ligatures (multi-characters) that are unique across languages.
So, we needed to refactor the string logic to be language specific (and essentially Gen1 / Gen2 specific as well). Now, we have APIs that can accept language to select the correct text character encoding table. To use these APIs correctly, this required updating the Language/Version tracking of a Gen1/Gen2 save file, as this language will be used to get the right set of characters. Rather than assuming English (or detecting based on current nickname), we can do a little better by caching what language we think the save file is by looking at the file name. Granted, this doesn't work at all for Virtual Console saves (always named
sav.dat), so we also need to provide a program setting to allow users to specify a forced value if none is detected from the filename.Extending this logic to Generation 3, we can also cache the assumed language & version for these files as well. Helps eliminate the "what version is this" popup for FireRed/LeafGreen, which is needed to show the correct stats & sprite for Deoxys. Players can still manually change it with the Block Editor, and if none is detected from File Name, they can still answer the popup. Might be problematic if people randomly name their save files and hit any of the recognition patterns, but that's on them for naming things weirdly.
Since our brains are in Gen1/2 mode, we can also resolve the issue / old cruft of
PokeList1/PokeList2. In these GameBoy save files, the game stores each box as a serialized list of entities, rather than just raw slot data like future games. When PKHeX first implemented Gen1/2 support, the decision was made to "unpack" these box objects into raw slot data, so that the existing APIs can read/write data without any extra abstraction. The code was implemented and basically remained unchanged for 6+ years, as it did its job.When looking at the logic, it creates a new list object, loads all PKM from the box-list, then writes all PKM individually to a hidden buffer used for the raw slot data API. We can do a little better by just copying directly, no allocation required. Additionally, we can try to retain the original data (if uninitialized && unaltered by the user during editing), which allows for the program to be used without disturbing any set-up glitches.