search

aetherism / dios-mios-lite-source-project

Automatically exported from code.google.com/p/dios-mios-lite-source-project
Other
0 stars 0 forks source link

SD card corruption? #26

Open GoogleCodeExporter opened 8 years ago

GoogleCodeExporter commented 8 years ago 
While trying out DML, I've hit problems where the SD card filesystem has been 
corrupted. The first time, it caused both FATs to be unusable. (This was 
recovered with TestDisk.) The second time, it caused various directories and 
files to be unreadable.

I'm not sure if this is caused by DML logging data to the SD card or what.

I will test DML again with a small 1GB SD card and Mario Kart: Double Dash!! to 
see exactly what's being overwritten.

Original issue reported on code.google.com by gerbilsoft on 19 Feb 2012 at 6:37

GoogleCodeExporter commented 8 years ago 
Also, this was with DML r46, compiled with devkitARM r36.

Original comment by gerbilsoft on 19 Feb 2012 at 6:37

GoogleCodeExporter commented 8 years ago 
The loader I used was USB Loader GX r1154. I have no idea if that might be 
what's causing the problem. I'll test it again with both USB Loader GX and 
NeoGamma (R9b56).

Original comment by gerbilsoft on 19 Feb 2012 at 6:53

GoogleCodeExporter commented 8 years ago 
Can you format your card, and then repeat whatever corrupted your sd card? The 
log file DML creates in this case might be helpful.

And which game did corrupt your sd card?

Original comment by nintendo...@kaffeeschluerfer.com on 19 Feb 2012 at 10:37

GoogleCodeExporter commented 8 years ago 
r47 might fix this problem, maybe not.

Original comment by nintendo...@kaffeeschluerfer.com on 19 Feb 2012 at 2:24

GoogleCodeExporter commented 8 years ago 
I repaired the filesystem (recovered files with TestDisk and reformatted). I'll 
try again with both r46 and r47.

The games I tested were Mario Party 7, Sonic Adventure DX, Sonic Adventure 2 
Battle, and Super Smash Bros. Melee. The most obvious corruption was when 
BootMii's files were trashed. This either resulted in the BootMii menu not 
showing up, or in the disc LED flashing twice on startup, indicating BootMii 
couldn't find the boot files at all.

(Notably, starting SADX seemed to be the most unreliable, and it once resulted 
in the BootMii files being trashed.)

Original comment by gerbilsoft on 19 Feb 2012 at 8:27

GoogleCodeExporter commented 8 years ago 
Test results with Mario Kart: Double Dash:
- r46: Corrupts some files.
- r47: Same issue.

I thought the problem may have been related to shutting down the console from 
the interrupt handler instead of simply setting a flag to let the main function 
know it needs to shut down, but this doesn't appear to be the case. It still 
gets corrupted. (Using the flag is still a good idea, and I'll submit patches 
for that once I figure out what's going on here.)

I'm going to try disabling SD writes and see if that fixes anything. I'm 
suspecting something is misbehaving in the FAT32 code. (Of note, it didn't even 
try to boot the game if the SD card wasn't partitioned with DOS compatibility; 
that is, the first sector of the partition == sector 63; and it corrupted the 
filesystem anyway.)

Original comment by gerbilsoft on 26 Feb 2012 at 2:39

GoogleCodeExporter commented 8 years ago 
With r47, I'm now getting this problem in fsck:

/DM.LOG
  File size is 3 bytes, cluster chain length is 0 bytes.
  Truncating file to 0 bytes.

It doesn't appear to be corrupting anything else right now. I'm suspecting that 
the FAT filesystem module doesn't like some aspect of my 1GB SD card, because 
with my 16GB SDHC card, logging worked fine.

Disabling SD write support did prevent the corruption, but it also obviously 
prevents logs from being written.

(On another note: DML Installer v1.1 doesn't properly update both FATs, which 
resulted in some filesystem corruption earlier. This was easily fixed by 
running fsck after using DML Installer.)

Original comment by gerbilsoft on 26 Feb 2012 at 2:53

GoogleCodeExporter commented 8 years ago 
With NMM r44 if you save Luigi Mansion after turn of wii and turn on again load 
Luigi Mansion, press start and gives: The save file are corrupted.

Original comment by ilyassta...@gmail.com on 26 Feb 2012 at 11:47

GoogleCodeExporter commented 8 years ago 
I'd be very happy if you do something to save super mario sunshine in NMM in 
real nand.

Please

Thanks

Original comment by igordima...@bol.com.br on 26 Feb 2012 at 3:06

GoogleCodeExporter commented 8 years ago 
I have tried to save super mario sunshine in NMM in pal realnand but it gives: 
The device in Slot A is not supported" error.

Original comment by ilyassta...@gmail.com on 26 Feb 2012 at 5:08

GoogleCodeExporter commented 8 years ago 
And how can I fix super mario sunshine save.

Original comment by ilyassta...@gmail.com on 26 Feb 2012 at 5:10

GoogleCodeExporter commented 8 years ago 
and there is no way to fix the problem "The device in Slot A is not supported" 
of super mario sunshine?

Original comment by igordima...@bol.com.br on 26 Feb 2012 at 5:18

GoogleCodeExporter commented 8 years ago 
I'd like to point out that the specific issue I'm having has nothing to do with 
NMM. It happens with plain DML without memory card emulation.

Original comment by gerbilsoft on 26 Feb 2012 at 7:26

GoogleCodeExporter commented 8 years ago 
ok,
thanks a lot

Original comment by igordima...@bol.com.br on 26 Feb 2012 at 7:54

GoogleCodeExporter commented 8 years ago 
I tried installing a clean MIOS and then reinstalling r47, and here's what I 
get with my 1GB SD card after running MKDD:

Formatted as FAT32, 4 KB clusters:

gs_laptop /home/david/Wii/Homebrew # fsck /dev/sdb1
fsck from util-linux 2.19.1
dosfsck 3.0.9, 31 Jan 2010, FAT32, LFN
/DM.LOG
  Contains a free cluster (120321). Assuming EOF.
/DM.LOG
  File size is 3 bytes, cluster chain length is 0 bytes.
  Truncating file to 0 bytes.
Leaving file system unchanged.
/dev/sdb1: 25 files, 120318/249484 clusters

Formatted as FAT16, 32 KB clusters: ..actually, the corruption "disappears", 
but the beginning of DM.LOG shows garbage.

So it seems like there's two problems:
- Using clusters smaller than 32 KB is completely broken.
- With 32 KB clusters, DM.LOG shows data corruption.

My 16 GB SDHC card used 32 KB clusters, so it must have overwritten some other 
area as well. This might be fixed in r47; I'll have to test it again.

================================================================

dm.log from the first run using FAT16 with 32 KB clusters:

���'[�ߌ��D8{p������������,�p�=\����
��.�7������q��5p?�u>�ne�-;XOޝ#����
                                                                   g\P�����ܦW�u*����w*(��w�`�ܝ����|�̲|dž#a���%1-�趨��P�����#�I�>!8����%�ӻ��.��u{m-Ĵ�qю?�s���Q�i�K�\�[)��n��X����Ǩ�k�+���;�5�m���ӣ���+�Zq�����a���1������A>Z�Z"6+�<�{R7����~@g��KC��ɦ\㹺9br�z;�Sls����|1����9��$��l�}����_bg���1�I$�y1�7��8H$|D)��������Yo�0�yLzu�U��e�Xf;)�S}���O������יB��ϻ!�>�Mz1�@G�<��/n=�FI�8�S�uns����;�#�+07�j�"|��R��S����]J^�;+ܗv����H`�s�Nu�Kۇe{����▒kK�I�(][1�`p|�pyW�▒��uAT�Ӊ{���O���8
\Q�r�?���dnX�>�W{���v�v?�V�J���q�x
¼�׿��������~�{�o���Z`�����������
����{����{��콭����w���O�Kߟ�o�{W���
wy���t������7yS�g~s���?:�nw��Z�z�S����
����������������������w�������
���f���������w�W�y����������{���
���ԋ���{��g3����������e���_����
|��������������g^���|���������
����1�i������������������}�]_���
���������������������۩����s��
��i������k��~���?w��U?���������o
y��R'm��켭����kyg��������ʷ��������
�������]�~~����^����[�[��?����r���
�{����v������}?����4��ˊ���Ɲ�����
����yk�x��u��}�����_|�~������}����
'������~}�����<��?��
                                                              7!�V��{��\ns����x�P���f�0�NTsP@tP%H���4����D�tP%H���4�����#rZM�@��`�tP%H���4����#�yB�"rZ0eyBw��YyS7�g����������������h$���gPatch:Found [SetInterruptMask]: 0x800E8ABC
Patch:Found [__OSDispatchInterrupt]: 0x800E8CDC 0x800E8E84
Patch:[Hook:OSSleepThread] at 800EC188
Using entry:0 to alloc 128 bytes...
Patch:Failed to open/find cheat file:"/games/GM4E01/GM4E01.gct"
Patch:Found [DVDLowReadAudio]: 0x800AB20C
Patch:Found [DVDLowAudioStatus]: 0x800AB2A4
Patch:Found [DVDLowAudioConfig]: 0x800AB330
Patch:Found [DVDInquiryAsync]: 0x800ADD38
Patch:Found [AIResetStreamSampleCount]: 0x800B0B9C
Patch:Found [__GXSetVAT]: 0x800BCF2C
Patch:Found [__fwrite B]: 0x801024F0
ask]: 0x800E8ABC
Patch:Found [__OSDispatchInterrupt]: 0x800E8CDC 0x800E
  Game Name ... GM4E
  Company ..... 01
  Disk # ...... 0
  Game ver .... 0
  Streaming ... OFF
  Country ..... US

  Banner ...... Available

================================================================

dm.log from the second run using FAT16 with 32 KB clusters:

_B�DIOS-MIOS Lite
Built: Feb 27 2012 20:17:59
This software is licensed under GPLv3, for more details visit:
http://code.google.com/p/dios-mios-lite-source-project
DVD:Error:04020400
Using entry:0 to alloc 1024 bytes...
Using entry:1 to alloc 32 bytes...
;�5�m���ӣ���+�Zdx賱�Ԇ���,Dl��٘N遹T6���d
�$$vn#riSRAM:Mode:0(0) EURGB60:0 Prog:1
SRAM:Mode:0(0) EURGB60:0 Prog:1
ɦ\㹺9br�z;�Sls����|1����9��$��l�}����_bg
���1�I$�y1�7��8H$|D)������<< RVL_SDK - SI       
release build: Jun 23 2006 16:12:58 (0x4200_60422) >>

Revolution OS
Kernel built : Jun 23 2006 16:12:26
Console Type : Retail 2
Firmware     : 0.0.0 (0/0/2000)
Memory 24 MB
MEM1 Arena : 0x1338480 - 0x1700000
MEM2 Arena : 0x7fffffff - 0x0
<< RVL_SDK - OS         release build: Jun 23 2006 16:12:26 (0x4200_60422) >>
Now in compatible mode.
<< RVL_SDK - VI         release build: Jun 23 2006 16:13:04 (0x4200_60422) >>
Bus  Clock = 162000000 Hz
Core Clock = 486000000 Hz
<< RVL_SDK - PAD        release build: Jun 23 2006 16:12:54 (0x4200_60422) >>
<< RVL_SDK - DVD        release build: Jul  6 2007 16:34:22 (0x4199_60831) >>

--- GAMECUBE BOOTROM for REVOLUTION v1.1 ---
Loading....DOLPHIN LAYOUT FORMAT
4 MB
MEM1 Arena : 0x1338480 - 0x  appLoaderFunc1  ...... 0x1200258
xApploader Initialized.  $Revision: 32 $.
This Apploader built Apr 17 2003 12:46:20

Apploader Initialized
K - VI  release build: Jun 23 2006 16:13:04 (0x4200_60422) >>
Bus  Clock = 162000000 Hz
Core Clock = 486000Patch:Found [__DVDIntrruptHandler]: 0x800AA768
Patch:[__DVDInterruptHandler] 800AA784
Patch:[__DVDInterr�x�P���f�0�NTsP@tP%H���4����D�tP
%H���4�����#rZM�@��`�tP%H���4����#�yB�
"rZ0eyBw��YyS7�g����������������h$����
�gPatch:Found [SetInterruptMask]: 0x800E8ABC
Patch:Found [__OSDispatchInterrupt]: 0x800E8CDC 0x800E8E84
Patch:[Hook:OSSleepThread] at 800EC188
Using entry:0 to alloc 128 bytes...
Patch:Failed to open/find cheat file:"/games/GM4E01/GM4E01.gct"
Patch:Found [DVDLowReadAudio]: 0x800AB20C
Patch:Found [DVDLowAudioStatus]: 0x800AB2A4
Patch:Found [DVDLowAudioConfig]: 0x800AB330
Patch:Found [DVDInquiryAsync]: 0x800ADD38
Patch:Found [AIResetStreamSampleCount]: 0x800B0B9C
Patch:Found [__GXSetVAT]: 0x800BCF2C
Patch:Found [__fwrite B]: 0x801024F0
ask]: 0x800E8ABC
Patch:Found [__OSDispatchInterrupt]: 0x800E8CDC 0x800E
  Game Name ... GM4E
  Company ..... 01
  Disk # ...... 0
  Game ver .... 0
  Streaming ... OFF
  Country ..... US

  Banner ...... Available

Original comment by gerbilsoft on 28 Feb 2012 at 1:45

GoogleCodeExporter commented 8 years ago 
Some more test results:
- With 16 KB clusters, DML won't start at all; the left side of the screen has 
a few vertical purple bars.
- With 64 KB clusters, it seems to work, and dm.log is generated (2417 bytes), 
but it's all 00 bytes.
- I noticed that vsprintf.c uses vsprintf() / sprintf(), which are unsafe 
functions because they don't do bounds checking. I converted them to 
vsnprintf() / snprintf(), but unfortunately that didn't fix the corruption 
problems. I'll post these patches later this week.

Original comment by gerbilsoft on 29 Feb 2012 at 4:03

GoogleCodeExporter commented 8 years ago 
Today I was going over the DML code, and I noticed that the bounce-buffer used 
for SDHCI transfers to the Starlet SRAM uses a buffer at 0x1000. This is 
actually used by the Dolphin OS. I moved the buffer to 0x2D00 (which reduced 
the patch code area by 512 bytes), and it works a little better.

The overall issue seems to be that dbgprintf() is being called simultaneously 
from two different points, since I see some overlapping writes from the Dolphin 
OS and from DML. How is the Dolphin OS fwrite handler implemented? If it's an 
interrupt, that may be what's causing the issue.

The better question, of course, is why does it seem like I'm the only one who's 
hitting SD card corruption in DML :(

================================================================

Current DM.LOG file:

DIOS-MIOS Lite
Built: Feb 29 2012 21:11:13
This software is licensed under GPLv3, for more details visit:
http://code.google.com/p/dios-mios-lite-source-project
DVD:Error:04020400
Using entry:0 to alloc 1024 bytes...
Using entry:1 to alloc 32 bytes...
SRAM:Mode:0(0) EURGB60:0 Prog:1
SRAM:Mode:0(0) EURGB60:0 Prog:1
<< RVL_SDK - SI         release build: Jun 23 2006 16:12:58 (0x4200_60422) >>

Revolution OS
Kernel built : Jun 23 2006 16:12:26
Console Type : Retail 2
Firmware     : 0.0.0 (0/0/2000)
Memory 24 MB
MEM1 Arena : 0x1338480 - 0x1700000
MEM2 Arena : 0x7fffffff - 0x0
<< RVL_SDK - OS         release build: Jun 23 2006 16:12:26 (0x4200_60422) >>
Now in compatible mode.
<< RVL_SDK - VI         release build: Jun 23 2006 16:13:04 (0x4200_60422) >>
Bus  Clock = 162000000 Hz
Core Clock = 486000000 Hz
<< RVL_SDK - PAD        release build: Jun 23 2006 16:12:54 (0x4200_60422) >>
<< RVL_SDK - DVD        release build: Jul  6 2007 16:34:22 (0x4199_60831) >>

--- GAMECUBE BOOTROM for REVOLUTION v1.1 ---
Loading....DOLPHIN LAYOUT FORMAT
4 MB
MEM1 Arena : 0x1338480 - 0x  appLoaderFunc1  ...... 0x1200258
xApploader Initialized.

Original comment by gerbilsoft on 1 Mar 2012 at 3:12

GoogleCodeExporter commented 8 years ago 
Bah, that log got truncated.

DIOS-MIOS Lite
Built: Feb 29 2012 21:11:13
This software is licensed under GPLv3, for more details visit:
http://code.google.com/p/dios-mios-lite-source-project
DVD:Error:04020400
Using entry:0 to alloc 1024 bytes...
Using entry:1 to alloc 32 bytes...
SRAM:Mode:0(0) EURGB60:0 Prog:1
SRAM:Mode:0(0) EURGB60:0 Prog:1
<< RVL_SDK - SI         release build: Jun 23 2006 16:12:58 (0x4200_60422) >>

Revolution OS
Kernel built : Jun 23 2006 16:12:26
Console Type : Retail 2
Firmware     : 0.0.0 (0/0/2000)
Memory 24 MB
MEM1 Arena : 0x1338480 - 0x1700000
MEM2 Arena : 0x7fffffff - 0x0
<< RVL_SDK - OS         release build: Jun 23 2006 16:12:26 (0x4200_60422) >>
Now in compatible mode.
<< RVL_SDK - VI         release build: Jun 23 2006 16:13:04 (0x4200_60422) >>
Bus  Clock = 162000000 Hz
Core Clock = 486000000 Hz
<< RVL_SDK - PAD        release build: Jun 23 2006 16:12:54 (0x4200_60422) >>
<< RVL_SDK - DVD        release build: Jul  6 2007 16:34:22 (0x4199_60831) >>

--- GAMECUBE BOOTROM for REVOLUTION v1.1 ---
Loading....DOLPHIN LAYOUT FORMAT
4 MB
MEM1 Arena : 0x1338480 - 0x  appLoaderFunc1  ...... 0x1200258
xApploader Initialized.  $Revision: 32 $.
This Apploader built Apr 17 2003 12:46:20

Apploader Initialized
K - VI  release build: Jun 23 2006 16:13:04 (0x4200_60422) >>
Bus  Clock = 162000000 Hz
Core Clock = 486000Patch:Found [__DVDIntrruptHandler]: 0x800AA768
Patch:[__DVDInterruptHandler] 800AA784
Patch:[__DVDInterruptHandler] 800AA79C
Patch:[__DVDInterruptHandler] 800AA7C4
Patch:Found [DVDLowRead]: 0x800AAB28
Patch:[cbForStateBusy] 800AD55C
MB
MEM1 Arena : 0x1338480 - 0x  appLoaderFuPatch:Found [__OSDispatchInterrupt]: 
0x800E8CDC 0x800E8E84
Patch:[Hook:OSSleepThread] at 800EC188
Using entry:0 to alloc 128 bytes...
Patch:Failed to open/find cheat file:"/games/GM4E01/GM4E01.gct"
Patch:Found [DVDLowReadAudio]: 0x800AB20C
Patch:Found [DVDLowAudioStatus]: 0x800AB2A4
Patch:Found [DVDLowAudioConfig]: 0x800AB330
Patch:Found [DVDInquiryAsync]: 0x800ADD38
Patch:Found [AIResetStreamSampleCount]: 0x800B0B9C
Patch:Found [__GXSetVAT]: 0x800BCF2C
Patch:Found [__fwrite B]: 0x801024F0
- 0x  appLoaderFuPatch:Found [__OSDispatchInterrupt]: 0x800E8CDC 0x800E
  Game Name ... GM4E
  Company ..... 01
  Disk # ...... 0
  Game ver .... 0
  Streaming ... OFF
  Country ..... US

  Banner ...... Available

Original comment by gerbilsoft on 1 Mar 2012 at 3:12

GoogleCodeExporter commented 8 years ago 
Okay, I have an initial set of patches that should help a bit, but they don't 
seem to fix the corruption issues.

Patch 1: Convert (v)sprintf() to (v)snprintf() - prevents buffer overflows if 
the formatted data exceeds 127 bytes.
Patch 2: memcpy()'s second parameter should be const void*, not void*.
Patch 3: sdmmc_write()'s third parameter should be const void*, not void*.
Patch 4: vsprintf.c: Allocate the buffer on the stack instead of on the heap.
Patch 5: Patches.c: Cleaned up memory allocation values for the cheathook.

Patch 1 is the most important, since it prevents buffer overflows.

Patches 2 and 3 fix some gcc warnings, and aren't too important.

Patch 4 may fix some reentrancy issues if dbgprintf() is called from an 
interrupt handler while dbgprintf() is already running. (This seems to be the 
actual problem, since I'm seeing the mixed-up dbgprintf() stuff; however, I 
can't pinpoint the exact issue.)

Patch 5 cleans up all the values for CheatHook and uses macros instead of 
"magic" numbers, which should make it easier to update in the future.

(The patches were stored in my local git-svn repository, which uses my Verizon 
email address instead of git. You don't need to include the email info in the 
svn commits.)

Original comment by gerbilsoft on 5 Mar 2012 at 1:00

Attachments:

GoogleCodeExporter commented 8 years ago 
Forgot to add patches 3, 4, and 5.

Original comment by gerbilsoft on 5 Mar 2012 at 1:00

Attachments:

GoogleCodeExporter commented 8 years ago 
I found a minor error in patch 5 regarding the cheathook code length, so I 
fixed that in another patch. (The code length was off by 8 bytes.)

Original comment by gerbilsoft on 5 Mar 2012 at 2:57

Attachments:

GoogleCodeExporter commented 8 years ago 
Any word on applying these patches? (At least the first four should be 
applied...)

If there's any issues applying them because of recent commits, let me know so I 
can rebase them.

Original comment by gerbilsoft on 6 Mar 2012 at 5:32

GoogleCodeExporter commented 8 years ago 
I'm seeing CHKDSK report errors on my FAT32 w/ 64KB clusters 32GB SDHC card as 
well:
"Convert lost chains to files (Y/N)? Y
64 KB in 1 recovered files.
Windows has made corrections to the file system."

It looks like the cluster was all zeroes. (This is a new card though, so it may 
be that it was never written to rather than something writing all zeroes to it).

I'm not totally sure if this is related to DML or not though. Nothing actually 
seems to be broken. I'll post again if I see any further errors.

I did notice dm.log has no modified timestamp, is that intentional?

Original comment by ssj4andr...@gmail.com on 21 Mar 2012 at 11:52

GoogleCodeExporter commented 8 years ago 
Looks like the FAT driver (ff.c) is not re-entrant.
The "#define _FS_REENTRANT   0" could be changed to "#define _FS_REENTRANT   1".
However it looks like long file names must also be disabled then. It may break 
other things as well since the FS calls just fail if there's already an 
operation in progress.
The easiest thing to do would be to disable writes to the SD card while in an 
interrupt handler. I'm assuming there's some way to detect if we're in an 
interrupt?
I suppose the proper way to do it would be to write a buffering mechanism for 
log output, only flushing it to file when not in an interrupt.
BTW, do you happen to know if any other FS calls are made from interrupts?

Original comment by ssj4andr...@gmail.com on 22 Mar 2012 at 1:27

GoogleCodeExporter commented 8 years ago 
I rebased the patches to r56, and managed to "fix" the corruption by disabling 
logging to SD entirely. Obviously this isn't the best solution, but it works 
for now.

Here's the updated vsnprintf() patches. I didn't include a patch to disable 
logging to SD, but it's easy enough to do that by editing vsprintf.c:

diff --git a/DML/vsprintf.c b/DML/vsprintf.c
index 317fa67..054115a 100644
--- a/DML/vsprintf.c
+++ b/DML/vsprintf.c
@@ -495,7 +495,7 @@ int dbgprintf( const char *fmt, ...)
                GeckoSendBuffer( buffer );
        }

-       if (dbgprintf_sd_access)
+       if (0 && dbgprintf_sd_access)
        {
                // Write all debug output to the log file, independent from the usb gecko output
                // Only fwrite output from games won't be logged to sd card

Original comment by gerbilsoft on 25 Mar 2012 at 3:16

Attachments: