search

DarkMatterCore / nxdumptool

Generates XCI/NSP/HFS0/ExeFS/RomFS/Certificate/Ticket dumps from Nintendo Switch gamecards and installed SD/eMMC titles.
GNU General Public License v3.0
984 stars 73 forks source link

Game RomFS dump is very slow #169

Closed BitBlitObviMormon closed 1 month ago

BitBlitObviMormon commented 1 month ago

Describe the bug I am trying to dump TOTK's RomFS, but it is dumping at 43KB/s. Other switch programs that write to the SD card copy data at over 100MB/s with this card. Is there anything I can do to help the program work faster? I'll be lucky if this finishes next week at this pace.

To Reproduce Steps to reproduce the behavior: Dump RomFS for The Legend of Zelda: Tears of the Kingdom

Info

Screenshots image

DarkMatterCore commented 1 month ago

Getting good speeds for sequential write operations on the SD card is plausible, but random access write operations are a completely different thing. The Switch, and more specifically, Nintendo's FS services, aren't exactly good at handling these scenarios (and they're not required to).

Games like BotW, TotK and ACNH use RomFS sections with lots of tiny files. Each time one of these files has to be dumped:

  1. The full directory tree used by the file's absolute path must be replicated on the SD card.
  2. The file must be created on the SD card.
  3. File data must be written to the SD card.
  4. The file must be closed and any pending data (e.g. stored in either a HW cache or a SW buffer) must be flushed to the SD card.

This process requires a lot of back and forth operations on the SD card's FAT volume. Multiply this by the terrific number of file entries available in any of the games I mentioned (among others), and the result is what you're currently experiencing.

This is not something that can be easily fixed -- all Switch homebrew applications rely on Nintendo's FS services to perform SD card I/O operations. And even then, we're also limited by the Switch's SDMMC interface capabilities, which aren't exactly the best either.

Simply put, the 100 MiB/s speed you're referring to is more than likely related to a sequential write operation, which can't really be compared to what nxdumptool is doing here.

If you want to get better speeds while dumping extracted NCA FS sections, I'd suggest switching to USB-based dumping -- either by connecting your Switch to a PC and using the Python-based host app, or connecting a USB mass storage device to your Switch and using that as the output storage instead.

BitBlitObviMormon commented 1 month ago

Thanks, this is a well-thought answer and I appreciate it.

DarkMatterCore commented 1 month ago

Thanks, this is a well-thought answer and I appreciate it.

Glad to help. Keep in mind you can cancel the ongoing process by holding the B button for ~3 seconds -- it'll initially look like it froze, but it'll actually be deleting the incomplete dump.

As for dumping data to a connected PC, you'll require Python 3.10+ and some Python dependencies -- as well as a libusbK driver if you're using Windows, or a udev rule if you're using Linux.

USB mass storage device support should be plug 'n play for the most part. FAT, NTFS and EXT volumes in MBR/GPT/EBR drives are all supported.