Can't get MSU-1 support to work

[shinji257]

From my understanding MSU-1 support should be working but I can't find any documentation on what the XML format should be to load it. When I get what I think is the correct format (based on research) the emulator just crashes and eventually closes itself when I tell it to continue (albeit without audio).

Am I doing something wrong with the format or is this a bug? Linked is my xml file.

This is for the Link to the Past randomizer.

https://gist.github.com/shinji257/3be89054b6a906c798f1d0a2bede7a71

[YoshiRulz]

If someone wants to try to repro this, put a breakpoint here (both rom_xml and xml should be not null): https://github.com/TASEmulators/BizHawk/blob/36be9e9fc760ed5f4d30407b0e89a10905cf4f0a/BizHawk.Emulation.Cores/Consoles/Nintendo/SNES/LibsnesApi_CMD.cs#L67

[shinji257]

So I went ahead and setup VS Community 2019 (no I'm not really a coder at all) and ran the code with the breakpoint set as you specified. At that breakpoint rom_xml has data but xml is showing up as null.

+       $exception  {"Object reference not set to an instance of an object."}   System.NullReferenceException
+       this    {BizHawk.Emulation.Cores.Nintendo.SNES.LibsnesApi}  BizHawk.Emulation.Cores.Nintendo.SNES.LibsnesApi
+       rom_xml {byte[2578]}    byte[]
+       rom_data    {byte[2097152]} byte[]
        xml null    string

[vadosnaprimer]

During the execution break, view the call stack and navigate to the function that calls CMD_load_cartridge_normal() (double click), maybe you can see why null arrives as an argument.

[YoshiRulz]

The argument isn't null, the GetString call failed.

edit: Tried repro, failed, didn't even hit the method in question (I'm guessing I need to actually have the game rom as well). As mentioned on IRC I have a feeling that replacing Encoding.ASCII w/ Encoding.UTF8 will solve the problem

[vadosnaprimer]

@shinji257 what value does rom_xml actually have, can you check? Hover on it in the code during break.

[shinji257]

Hovering over it shows 2578 individual decimal values. I did check though and the XML I posted earlier converts to these decimal values and mine is 2578 characters long including spaces.

Image so you can see the contents:

Something I have not done before. Non-debug stack trace of the crash: https://gist.github.com/shinji257/2b987c16c06a708a293fe0e7b4d6f976

What happens if I have (what is probably) an invalid xml format. This test I removed the name definition from the xml and loaded it. https://gist.github.com/shinji257/260ecb3f4221824c0c71fcb2606e7482

EDIT: Ok. This crash actually seems to be happening later. I'm stepping the code (sorry I'm a noob to this but trying to help) and GetString does succeed to convert it back or seems to as the very next step shows xml get populated.

System.NullReferenceException thrown at line 3733 in MainForm.cs.

EDIT: I've been backtracking on versions to see if MSU-1 support has ever worked and I don't see an instance where it seems to have worked at all. 1.5.0 was when it was added according to the changelog but I couldn't get it to work there (no cores can load this game). Later versions could load the XML but it failed to boot with the game indicating it couldn't find the data pack. My XML fails when (I guess) the game goes to try and initialize the music failing to do so and just stalling out. I've been using Super Road Blaster (the game specifically mentioned in the 1.5.0 changelog entry) as the test for this as it should be working at least.

NullReferenceException appeared between 2.2.2 and 2.3.0.

[jdpurcell]

I just fixed the NullReferenceException (edit: in 09dc1df9a --yoshi) but I still can't get that ROM working.

I see the XML contents getting passed in inside LibsnesApi.CMD_load_cartridge_normal but that's as far as I can debug for now.

[vadosnaprimer]

What are the testing steps for dummies?

[shinji257]

I'm using the Super Road Blaster MSU-1 pack that is searchable on Google (don't know if I can link it here...). It is specifically mentioned in the 1.5.0 changelog as the original basis for implementing the MSU-1 XML file parsing but I'm looking and I don't see any routine to actually ever load the PCM files. I may be blind though as I try to navigate the code.

[zeromus]

many MSU-1 archives are broken garbage that don't even conform to the real specs. I discussed this at least at http://tasvideos.org/Bizhawk/SNES.html and maybe elsewhere.

[boktai1000]

I've also been recently testing MSU-1 with BizHawk on a variety of different ROMs and all have been unsuccesful. I've attempted loading the XML files, or just the ROMs with the associated / appropriately named pcm files in the same folder and matching the name of the ROM to no avail.

• Attempting to load an .sfc file with the MSU files in the same directory causes the game to load regularly without the MSU files
• Attempting to load an .xml file with the files in the same directory provides the following error:

GBL load error
---------------------------
System.NullReferenceException: Object reference not set to an instance of an object.

   at BizHawk.Client.Common.RomLoader.LoadXML(String path, CoreComm nextComm, HawkFile file, String forcedCoreName, IEmulator& nextEmulator, RomGame& rom, GameInfo& game)
---------------------------
OK   
---------------------------

This is with BizHawk 2.8 on Windows 10 x64 21H1 - I've also tried to load an MSU-1 game that didn't include an XML file which is the recent translation of Tokimeki Memorial for SNES (Heartthrob Memorial: Under the Tree of Legends) to no avail. Link to the MSU-1 patch can be found here for testing: https://www.translated.games/snes/heartthrob

In all of my testing cases, whether it is user error or a problem with the BizHawk frontend, or my own settings - I was able to get all of the games to work properly in Snes9x 1.61 without really doing much, in fact I was able to point directly to the .sfc file in Heartthrob Memorial's case and it just worked. I know it was mentioned that some XML doesn't conform to the proper spec, but at least in my case I'm unable to get anything to work at all - it's possible that this is newer or different from when this report was initially filed.

[zeromus]

Seems this broke in 1.11.7 (1.11.6 works) Seems to me (I get different results from you) it's been broken in the same way ever since then (pink screen error message from the game). Maybe someone can bisect 1.11.6->1.11.7 and find out when it went wrong. Notably that was before it was waterboxed so it might be debuggable

[zeromus]

Well, bisecting this is complicated. But between those two versions we changed from x86 to x64 AND libsnes became waterboxed. it was probably the waterboxing that broke it. and the waterboxed libsnes can't be debugged. so... i dunno, I'm helpless with the waterboxed cores.

it seems possible that after waterboxing it, it was no longer able to read the needed auxilary files.

[boktai1000]

I attempted to try 1.11.6 and I'm getting the following error when attempting to load an XML (again, may be a malformed XML)

Note: Screenshots and error dialogue is taken from BizHawk 1.11.6 release just for this post

DGB load error
---------------------------
System.NullReferenceException: Object reference not set to an instance of an object.

   at BizHawk.Client.Common.RomLoader.LoadRom(String path, CoreComm nextComm, Boolean forceAccurateCore, Int32 recursiveCount)
---------------------------
OK   
---------------------------

XML Files tested: (this is the file I am attempting to load, but similar errors come from other XML files I have attempted to load

alttp_msu.xml

<?xml version="1.0" encoding="UTF-8"?>
<cartridge region="NTSC">
  <rom>
    <map mode="linear" address="00-6f:8000-ffff"/>
    <map mode="linear" address="80-ff:8000-ffff"/>
  </rom>

  <ram size="2000">
    <map mode="linear" address="70:0000-1fff"/>
  </ram>

  <msu1>
  <map address="00-3f:2000-2007"/>
  <map address="80-bf:2000-2007"/>
    <mmio>
      <map address="00-3f:2000-2007"/>
      <map address="80-bf:2000-2007"/>
    </mmio>
  </msu1>
</cartridge>

chrono_msu1.xml

<?xml version="1.0" encoding="UTF-8"?><cartridge region="NTSC">
    <rom>
        <map mode="shadow" address="00-3f:8000-ffff"/>
        <map mode="linear" address="40-7f:0000-ffff"/>
        <map mode="shadow" address="80-bf:8000-ffff"/>
        <map mode="linear" address="c0-ff:0000-ffff"/>
    </rom>
    <ram size="0x2000">
        <map mode="linear" address="20-3f:6000-7fff"/>
        <map mode="linear" address="a0-bf:6000-7fff"/>
        <map mode="linear" address="70-7f:0000-7fff"/>
    </ram>
    <msu1>
        <map address="00-3f:2000-2007"/>
        <map address="80-bf:2000-2007"/>
        <mmio>
            <map address="00-3f:2000-2007"/>
            <map address="80-bf:2000-2007"/>
        </mmio>
    </msu1>
</cartridge>

If I attempt to use the XML provided here https://gist.github.com/shinji257/3be89054b6a906c798f1d0a2bede7a71 though, it does seem to work. These XMLs provided with the releases I have give the aforementioned errors.

[zeromus]

There's no ROM NAME in there. The specifications for MSU-1 were a ^!*#(@ing abortion from the very outset. Is it required.... or not????????????? make <rom> element look like this <rom name="alltpwhatever.sfc" size="0x1234567"> not sure if size must be set not sure about any of this

needless to say, I have no interest in supporting this abortion of a file format, especially when it's no longer functioning in any event due to waterbox and consequently no longer debuggable

good luck, gentlemen

[boktai1000]

My apologies, I was simply trying to include any relevant information that may help with debugging and am unfamiliar with the specifications, as I'm more on the user end of things trying to help report things for a better end user experience rather than debugging and development work. I do appreciate the help and sharing of information, as you mention it isn't well documented and all over the place.

Cheers and hopefully it will help anyone else that looks to assist with it in the future