Broken files can be generated under specific conditions

[sp193]

Sometimes, ps2-packer will generate files that will not unpack properly. It rarely occurs and the problem seems to shift about, depending on the size of the built ELF.

It was already there in 2015, so I don't know when it started to happen (I always thought it was there). Anyway, I noticed that nobody wrote an issue entry for it, so here it is.

For example, this is a failed unpacking of OPL (forgot what version, but I kept the bad file and it is dated 22 January 2015):

dsedb S> dr
 at=70030c00  v0-1=00000001,0000001c  a0-3=01d00008,00000000,00000001,80075404
 t0-7=1fe01be0,00000001,00000001,00000001, 00000005,a0000000,00ff0000,0000001f
 s0-7=01c46c82,00100010,01c46c60,001ba68c, 002ba68c,00000000,00000000,00000000
 t8=80023900 t9=01d00024   k0=80016120 k1=00000000   gp=0010d3f0 sp=01ffecf0
 fp=00000000 ra=01d00178   lo=0000003c hi=00000000   sa=00000006 PC=01d00188
 badvaddr=a0000000 badpaddr=ff9fabf0
 $cause   = 0x70008010 [ BD2 CE3 EXC2=Reset IP7 EXC="Address Error (load/fetch)" ]
 $status  = 0x70030c13 [ Cu210 EDI EIE IM3 IM2 KSU=User EXL IE ]
  0x01d00180: 0x01896023  subu    $t4,$t4,$t1
  0x01d00184: 0x022b6823  subu    $t5,$s1,$t3
->0x01d00188: 0x91a90000  lbu     $t1,0($t5)
  0x01d0018c: 0x25ad0001  addiu   $t5,$t5,1
  0x01d00190: 0xa2290000  sb      $t1,0($s1)
  0x01d00194: 0x00000000  nop
  0x01d00198: 0x258cffff  addiu   $t4,$t4,-1

dsedb S> dt -v
<<tid   0  prio 128 count 2 status [Ready]  wakeup 0>>
pc    = 00081fd8  
func  = 00081fc0  
args  = 00000000  argc = 00000000  end of heap = 00000000
stack = 00000000  stackpointer = fffffd80  stacksize = 00000000
zero($00) = 00000000_00000000_00000000_00000000   $fpr0  = +0
  at($01) = 00000000_00000000_00000000_00000000   $fpr1  = +0
  v0($02) = 00000000_00000000_00000000_00000000   $fpr2  = +0
  v1($03) = 00000000_00000000_00000000_00000000   $fpr3  = +0
  a0($04) = 00000000_00000000_00000000_00000000   $fpr4  = +0
  a1($05) = 00000000_00000000_00000000_00000000   $fpr5  = +0
  a2($06) = 00000000_00000000_00000000_00000000   $fpr6  = +0
  a3($07) = 00000000_00000000_00000000_00000000   $fpr7  = +0
  t0($08) = 00000000_00000000_00000000_00000000   $fpr8  = +0
  t1($09) = 00000000_00000000_00000000_00000000   $fpr9  = +0
  t2($10) = 00000000_00000000_00000000_00000000   $fpr10 = +0
  t3($11) = 00000000_00000000_00000000_00000000   $fpr11 = +0
  t4($12) = 00000000_00000000_00000000_00000000   $fpr12 = +0
  t5($13) = 00000000_00000000_00000000_00000000   $fpr13 = +0
  t6($14) = 00000000_00000000_00000000_00000000   $fpr14 = +0
  t7($15) = 00000000_00000000_00000000_00000000   $fpr15 = +0
  s0($16) = 00000000_00000000_00000000_00000000   $fpr16 = +3.92364e-44
  s1($17) = 00000000_00000000_00000000_00000000   $fpr17 = +0
  s2($18) = 00000000_00000000_00000000_00000000   $fpr18 = +0
  s3($19) = 00000000_00000000_00000000_00000000   $fpr19 = +0
  s4($20) = 00000000_00000000_00000000_00000000   $fpr20 = +1.4013e-45
  s5($21) = 00000000_00000000_00000000_00000000   $fpr21 = +0
  s6($22) = 00000000_00000000_00000000_00000000   $fpr22 = -nan
  s7($23) = 00000000_00000000_00000000_00000000   $fpr23 = -nan
  t8($24) = 00000000_00000000_00000000_00000000   $fpr24 = +0
  t9($25) = 00000000_00000000_00000000_00000000   $fpr25 = +0
  gp($28) = 00000000_00000000_00000000_00000000   $fpr26 = +0
  sp($29) = 00000000_00000000_00000000_00000000   $fpr27 = +0
  fp($30) = 00000000_00000000_00000000_00000000   $fpr28 = +0
  ra($31) = 00000000_00000000_00000000_00000000   $fpr29 = +0
hi  = 00000000_00000000  lo  = 00000000_00000000  $fpr30 = +0
hi1 = 00000000_00000000  lo1 = 00000000_00000000  $fpr31 = +0
fcr31 = 01000001  facc = +0  sa = 00000000
<<tid   1  prio   0 count 4 status [Ready]  wakeup 0>>
pc    = 01d00008  
func  = 01d00008  
args  = 80017608  argc = 00000001  end of heap = 01fdf000
stack = 01fdf000  stackpointer = 01ffe770  stacksize = 00020000
zero($00) = 00000000_00000000_00000000_00000000   $fpr0  = +0
  at($01) = 00000000_00000000_ffffffff_80020000   $fpr1  = +0
  v0($02) = 00000000_00000000_00000000_00000002   $fpr2  = +0
  v1($03) = 00000000_00000000_00000000_00000104   $fpr3  = +0
  a0($04) = 00000000_00000000_00000000_00000002   $fpr4  = +0
  a1($05) = 00000000_00000000_00000000_00000185   $fpr5  = +0
  a2($06) = 00000000_00000000_00000000_00000004   $fpr6  = +0
  a3($07) = 00000000_00000000_00000000_00000044   $fpr7  = +0
  t0($08) = 00000000_00000000_00000000_00000000   $fpr8  = +0
  t1($09) = 00000000_00000000_ffffffff_a0023d50   $fpr9  = +0
  t2($10) = 00000000_00000000_00000000_00000040   $fpr10 = +0
  t3($11) = 00000000_00000000_ffffffff_a0023d00   $fpr11 = +0
  t4($12) = 00000000_00000000_00000000_201065c0   $fpr12 = +0
  t5($13) = 00000000_00000000_ffffffff_a0000000   $fpr13 = +0
  t6($14) = 00000000_00000000_00000000_00000004   $fpr14 = +0
  t7($15) = 00000000_00000000_00000000_0000001f   $fpr15 = +0
  s0($16) = 00000000_00000000_00000000_201065c0   $fpr16 = +0
  s1($17) = 00000000_00000000_00000000_00105600   $fpr17 = +0
  s2($18) = 00000000_00000000_00000000_00000010   $fpr18 = +0
  s3($19) = 00000000_00000000_00000000_00000200   $fpr19 = +0
  s4($20) = 00000000_00000000_00000000_00000001   $fpr20 = +0
  s5($21) = 00000000_00000000_00000000_00000000   $fpr21 = +0
  s6($22) = 00000000_00000000_00000000_01ffeac0   $fpr22 = +0
  s7($23) = 00000000_00000000_00000000_00000000   $fpr23 = +0
  t8($24) = 00000000_00000000_ffffffff_80023900   $fpr24 = +0
  t9($25) = 00000000_00000000_00000000_00019600   $fpr25 = +0
  gp($28) = 00000000_00000000_00000000_0010d3f0   $fpr26 = +0
  sp($29) = 00000000_00000000_00000000_01ffe9f0   $fpr27 = +0
  fp($30) = 00000000_00000000_00000000_01ffeac0   $fpr28 = +0
  ra($31) = 00000000_00000000_00000000_0010247c   $fpr29 = +0
hi  = 00000000_00000002  lo  = 00000000_00000000  $fpr30 = +0
hi1 = 00000000_0159f40e  lo1 = 00000000_00000000  $fpr31 = +0
fcr31 = 01000001  facc = +0  sa = 00000006

dsedb S> quit

The exception occurs at line 130 of n2e.s. It does not occur at the first iteration of the loop, but somewhere after.

I have this feeling that this is not actually where the problem is. I have been noticing tell-tale signs that the unpacker was executed repeatedly; I have stepped through this before and noticed that it will eventually jump to the EPC of the unpacked ELF. While stepping through that loop, I remember that eventually m_pos would become something like 0xFFFF00.

It's not a lot of details because I couldn't see why it happened and have no idea how to test it in detail, but I hope it'll do as a bug report.

[sp193]

I don't understand how getbit_8 translated into the assembly version of getbit. Why does bb seem to be in big-endian? What happened to the +1 in the expression? What is the logical OR of 0xFF0000 for?

[sp193]

For now, I am trying to see if it is related to the Signature and PackedELF data being too close to the instructions at _start.

From the EE restrictions documentation, having data adjacent to instructions may result in a wrong instruction being fetched, data destroyed or FPU division being affected, if the data & instructions have a certain pattern. A workaround is to add 5 or more nop/combination of nop & sync.p on the boundary between the data and instructions.

Since we don't know why "bad" files can be made, let's just see if this commit will make the problem disappear. Related commit: 80246fe7

[sp193]

It's been about 2 weeks since I have made the commit mentioned above, and I have not seen a single broken file being generated ever since. I guess it took care of the issue, so I'll be closing this ticket. Please feel free to reopen it if it pops up again.

As for an explanation why, I am quite convinced that it has to do with allowing data to exist so close to code... although I don't see any bit patterns exactly matching the descriptions under (17) and (18). However, the description for (2) does not specify any particular bit pattern.

This shows the 5 words before the code at _start, of one of those bad files:

0x4e4512ab 01001110 01000101 00010010 10101011
0x242c35e1 00100100 00101100 00110101 11100001
0x40254992 01000000 00100101 01001001 10010010
0x000000ff 00000000 00000000 00000000 11111111
0x3145324e 00110001 01000101 00110010 01001110 <- _signature
0x01c46c60 00000001 11000100 01101100 01100000

[sp193]

I got a broken file today. I also noticed today that there's this dummy.s file, which contains some data and is very close to code. I have removed it in 65a1c5d. I'll keep this issue open again for a while, to see if the bug comes back.

[Jay-Jay-OPL]

Just wondering if you plan to give us a new recompiled version: https://github.com/ps2dev/ps2-packer/releases