Save detection

[ghost]

Save type hardcoded to 32 KiB (256k) SRAM. Unknown how save type detection will be done.

Why not use a native_firm application to select a rom and write the path (to rom and save) and the save type to "open_agb_firm.ini"? Then trigger a reboot to open_agb_firm and it loads the ini file. I figure in native_firm there's more room to work with. You might even be able to write the config to ram since it isn't cleared on a reboot.

[profi200]

That's not what i had in mind. The goal was to completely decouple this from userland (pesky GBA VC "injects" and stuff) and make it optionally configurable/bootable from userland (but it should itself have a menu for configuration/game selection). The main way of running this should be through some bootloader. Luma's chainloader or fastboot3DS for example.

Flash memory hates writes. I don't think writing to the SD card every boot is a good idea especially writing less than erase block size. And space is not a problem currently. I still have large chunks of VRAM free and AXIWRAM is basically barely used.

I have talked with endrift (mGBA dev) about save detection and i think we agreed the best way is having a game database with at least save type and hashes of known good dumps. But then the problem is building this database properly. So i will probably resort to string search-based detection in the meantime which is not accurate in some cases.

[Kirit29]

how would this game database for saves work with modified roms aka rom hacks? As some change the save type. idk would per game settings be possible? or a check by game id?

[profi200]

If the save type is the same and the game code matches the original ROM it will probably detect it fine however with a hash mismatch. When the game code is changed there will be no other way than save override.

[Kirit29]

by game code do you mean the game id? ie BPRE for fire red? if so that can be changed too match by a hex edit

[ghost]

Ah fair enough.

Flash memory hates writes. I don't think writing to the SD card every boot is a good idea especially writing less than erase block size.

That's why I also suggested writing it to an area of ram that doesn't get overwritten on reboot. That would be more ideal.

[profi200]

Yes, i mean the 4 chars ID. It would be preferable if ROM hackers don't touch the ID. That would save everyone time.

Are there really cases where the ROM hack changes save type? Do they have a good reason to do that?

[Kirit29]

there are not many that change the save type its rare that that happens ive only seen it once.but there are many that add rtc to games that never have it ie. firered hacks.

[profi200]

Ugh, which is basically a save type change. Not sure if i can get away with assuming these games use the RTC enabled save type.

[Kirit29]

wouldn't just editing the game id via hex fix that? fore example bpre to axve. as that's ruby's id which uses a save type that's rtc enabled. that works for some emulators that have automatic rtc detection.

[profi200]

I can't expect noobs to pull out a hex editor to fix this by themself. It just doesn't work.

[Kirit29]

tbh i think the people in the rom hack playing community should know how to hex edit if they go through the process of of patching a rom with a computer much less installing a bootrom exploit on their 3ds to install cfw to run open agbfirm in the first place. and if necessary I can make a guide for you,for them, if it becomes a big enough problem that will explain the hex editing process.

[profi200]

Remember that not only experienced people play ROM hacks. I don't see many issues regarding this coming yet. At least hopefully.

My motivation to do anything is kinda down right now. I'm trying to do something useful to get the project forward but it barely works :/

[Kirit29]

No problem i understand fully. lack if motivation is a pain. Good luck on this project tho i will watch it closely as the 3ds is mainly what i use for gba emu and this would be a godsend. Not to mention a first in 3ds hackinglol. Take your time and work at your own pace

[profi200]

I mean. It's already fully playable if we ignore the missing save detection and file browser. I completed Metroid Zero Mission with this a few days ago and it was flawless. But if you compile it with a RTC enabled save type beware that it doesn't set the RTC properly yet.

I'm thinking about splitting this further into the project and a "lib3ds" of sorts. I wanted to do this for a long time with the fastboot3DS source (this is based on it) since it has code for almost everything now.

[Kirit29]

Unfortunately i can't compile it myself because my pc is out of my hands for awhile. Btw are u saying you can compile it with a different hardcoded save type? If so Im genuinely curious and would like to know how so i can get someone to do it for me ftm

[ghost]

Might I recommend checking out the source code for Gericom's GBARunner2? It's a similar project to yours and he's already got save detection working. (He also has the 3DS gyro working in GBA games! That's cool.) His code runs directly on the ARM9 and ARM7 so I figured you might be able to reuse some of it.

[Kirit29]

actually gbarunner2 uses some kind of special patches in place of genuine save type support iirc.

EDIT: Yup uses special sram patches according to gericom https://github.com/Gericom/GBARunner2/issues/40

[ghost]

EDIT: Yup uses special sram patches according to gericom Gericom/GBARunner2#40

Well it can't patch everything to sram. Pokémon saves fine in it. If it was sram patched the save wouldn't be big enough and it would get corrupted.

Edit: Yeah it looks like he isn't patching them to sram. He's patching them to some kind of custom save type.

https://github.com/Gericom/GBARunner2/commit/d6ae47220a3283db4b491d8dfb10fccb65082355

[ghost]

if i remember correctly pokemon can run with sram saves abiet with a few issues that are not gamebreaking

No the save corrupts when you beat the elite four and it'll throw up an error about the save being damaged every time you start the game. GBARunner2 doesn't cause this behavior. See that linked commit.

[Kirit29]

yeah i saw the linked commit lol. that's why i deleted my comment.

[profi200]

GBARunner2 works in a fundamentally different way to this. GBARunner2 runs in DS(i) mode and traps register accesses + emulating missing GBA hardware. This is basically a full GBA SoC implemented inside the 3DS SoC which means it has 100% compatibility (if we exclude GBA carts with extra hardware other than RTC and 64 MiB games). Also since this leaves ARM9/11 in 3DS mode we can use the 3DS GPU for scaling and the extra resolution of the screens.

The string search based save detection approach works but it's not 100% reliable which is why i wanted to have a database.

Save type can be changed here: https://github.com/profi200/open_agb_firm/blob/master/source/arm9/hardware/lgy.c#L51 https://github.com/profi200/open_agb_firm/blob/master/source/arm9/hardware/lgy.c#L74

[Kirit29]

ok ive figured out how to change the save size on my own but how dabout the type? @profii200

[profi200]

It's the GBA VC ROM footer values.

https://www.3dbrew.org/wiki/3DS_Virtual_Console#Config Below the table.

[Kirit29]

profii are you aware that the official register defs leaked recently? according to gericom they make the agb save registers more clear. Or is this old news to you lol if not i can send them to you

[profi200]

I have them since over a year (stripped down comments but they were not so important anyway). If you look closely at some code you will see it's "heavily inspired" by official headers.

[Kirit29]

i played pokemon emerald Enhanced last night on open agb_firm and it ran flawlessly and so did the rtc.

edit: nvm the rtc doesn't run after the console is turned off. just booted it up again and the time is the same as when i turned it of last night. edit you mentioned that the rtc doesnt set right. Do you have a plan on how to get it fully working? I'd be willing to test your efforts on that when the time comes and if it's need.

[profi200]

The RTC probably does run but it's reset to 1.1.2000 00:00 every boot. The biggest problem is probably that the 3DS side RTC is completely off unless you set it correctly with GodMode9. Other than that it's a matter of copying it from the 3DS RTC to the GBA one.

[Kirit29]

Profi is there somewhere else i can contact you? I got a few questions do you have a discord?

[profi200]

No, but i'm on the #GodMode9 IRC channel on Freenode (which has a bridge to the GodMode9 Discord).

edit: To be clear: The no was regarding Discord stuff.

[profi200]

In theory the RTC problem is solved now. Only day of week is not set but i don't know if this is used by any game.

[Kirit29]

@profi200 that's great. On the day of the week iirc the pokemon games use the day of the week in both vanilla and hacks for example in vanilla emerald there's a decoration shop that only appears on sunday and Thursday

Edit scratch that i was confusing gen 7 with gen3. but there might be others iirc

Edit 2: actually i think omitting day of the week would cause the berry glitch in r/s/e pokemon games as the berry system uses the date to tell berries to grow

[Kirit29]

btw how would i change the save type to flash 1m with rtc now. it seems you changed the format originally i had to put in 0xC but now is it SAVE_TYPE_FLASH_1M?

[profi200]

I don't know if day of week is used as said. It's a bit annoying to calculate especially because it's encoded as value 0-6 where it's unknown which is Monday.

And nothing changed. I just use aliases (to keep it simple) for the same values: https://github.com/profi200/open_agb_firm/blob/master/include/arm9/hardware/lgy.h#L12-L30

[Kirit29]

Well i know it uses the date for the berries when a berry is planted as the starting point. But i know for a fact rom hacks use the day of the week as you can change just the day of the week on your console and the events take place as scripted whether you go backwards or forwards in time

[profi200]

"change just the day of the week on your console"? At least in Pokémon games you can't just change time/date. Did you confuse that with the DS games? And changing the day also changes the date. What i mean is a counter from 0-6 that tells you which day of the week it is. Not day of month (0-31).

[Kirit29]

@profi200 what i mean is there is a tool that can add DNS aka Days nights and seasons that actually patches the games to read the date of your device. so say your 3ds says its thursday of x week of it will load any event u code to only happen on such day in game. For example in fire red you could code let say an npc to only appear at 7pm on a Thursday. I have the source code for the rtc code that is used for hacks if it would help

[Kirit29]

alright reviewing that source code of dns as far as i can see it seems it uses the date set at the start of the save to calculate day and time from then on every boot it offsets the time and day based on what i can see. it doesn't actually use the days of the week.

[profi200]

I don't know what "dns" is. Never heard of it. The original games not using day of week is at least good news (however that's just Pokémon). So i assume it's working?

[Kirit29]

Dns stands for days nights and seasons basically its a tool add code to a pokemon rom to make it use rtc when its doesn't usually like fire red for example @profi200

Also i haven't been able to test it out yet as im unable to build the firm. My pc is 30 miles away at my dad's and he's quarantined with the virus. I can test if you want but you'll have to send me a build. If that's ok with you. preferably SAVE_TYPE_FLASH_1m_SNO_RTC with save size of 128 kib

[profi200]

I opened my original Emerald cart and it looks like it's using Macronix flash which is what i have been using in my test with this game. Theoretically the save type doesn't matter as long as the size and addressing match (and the game doesn't check the flash ID). I found 1 little flaw with the current RTC handling. When starting a new game the Pokémon games reset the RTC to 1.1.2000 00:00:00 and if i save and reload the FIRM it jumps right back up to 2020. AgbBg solves this by using date/time offsets and saving them along the savegame. I can probably get away with this since mGBA does the same (just returns the current date/time).

open_agb_firm.zip

[Kirit29]

thanks btw was looking at gbatek for the rtc based functionality of the gba and nds then found this

Day of Week Register (septenary counter) 0-2 R/W Day of Week (00h..06h, custom assignment, usually 0=Monday?) 3-7 - Not used (always zero)

idk if it helps with your problem or not

[Dartz150]

So I haven't found any other place to discuss this, so in the mean time Iwill use this issue post. I have been testing this and so far is working to an usable state (I'm sure there are still many things to be added) and I'm pretty impressed on how it works far better than stock AGB_FIRM, congratullations! I see that save types and saving are not implemented yet but while reading this topic it seems that these are very close to be added, so I will ask only two things to continue my tests (I'm rebuilding this each time you push new commits btw).

1.- rom.gba at the root is a temporary dir right? I supose a file explorer will be added later, or will it be fronted based much like how TwilightMenu works with nds-bootstrap?

2.- It is possible to enable the BIOS intro? I have also noticed that if no rom is loaded, the bios boot animation shows up like half minute later.

3.- I see that the color display is not washed out and a bit darkened, which looks excelent! Does it use a LUT? Or some kind of shadder?

4.- Is it planned to add a customizable layout for the screen stretch and palette darkness?

Thank you in advance, and very sorry if I bottered you with these questions, this looks pretty good so far and I would love to continue testing this alongside GBARunner2 which is another amazing project.

[Kirit29]

@Dartz150 hey dartz its Dualbladedkirito from dsi mode hacking discord! You can change the save type but you have to recompile it every time for now. Change it with these https://github.com/profi200/open_agb_firm/blob/master/include/arm9/hardware/lgy.h#L12-L30 Here https://github.com/profi200/open_agb_firm/blob/master/source/arm9/hardware/lgy.c

[Dartz150]

Hey! Nice to see you here too @TriforceOfPwnage! Great, thanks for the tip, works as expected. As suggested, I do agree that a database should be used to determine the save type for the rom, and for romhacks, it should be fixed on the romhack side part as these aren't supported properly in AGB_FIRM and other hardware as well (mainly because these are mostly made for android emulators), or maybe an external editable database could be used for that, so depending on the romhack, a new entry should be writen in plain text, to tell which save type is used with said romhack, specially on the RTC fire red romhacks.

[Kirit29]

Hey rtc function properly in fire red rom hacks with this. Unlike regular agb_firm

[Dartz150]

Great to know! Another point for this over AGB_FIRM, then I see that the main holdback now is how to standardize a proper way to detect the save type chip right? See my comment above.

[Kirit29]

Pokemon adventure red doesn't boot tho. Green bottom screen and black top screen @profi200 do you know what would be causing this? I know that this game works on a gba flashcart on real hardware.

[Kirit29]

@Dartz150 also hes planning to make it have it's own rom select menu and be configurable within the firm iirc

[profi200]

@Dartz150

1. A file browser is planned.
2. Already implemented but hardcoded to off currently. Set this to true: https://github.com/profi200/open_agb_firm/blob/master/source/arm11/hardware/lgy.c#L39
3. It technically does use a shader but the PICA200 supports no custom shaders. It's just this as used by mGBA: https://gist.github.com/profi200/e31df7d31b8c2cdccc0608aaa71681fc#file-main-c-L75-L78
4. Maybe later. Without a proper citro3d port this is not easy to make configurable at runtime. Color lookup tables like on original GBA VC can be added easily though.

@TriforceOfPwnage ROM too big/SD card corruption? This is the ARM9 calling panic(). ROM size is limited to 32 MiB (hardware limitation). I should probably implement proper errors.

[Kirit29]

is there any way to trim the rom down abit via hex? its 1131 bytes over 32mib.