search

AwakeEchidna / wiiu-vc-extractor

Extracts Wii U Virtual Console roms from dumps created via DDD or dumped from the SNES Mini
GNU General Public License v3.0
66 stars 9 forks source link

Issue extracting SNES Street Fighter Alpha 2 (US) #50

Closed Lv36Blastoise closed 3 years ago

Lv36Blastoise commented 3 years ago

All my other games extracted fine, just having a little trouble on this one. I'm in no rush though, no pressure. I'm grateful this tool even exists, a lifesaver!! Annotation 2021-02-24 210913

wheatevo commented 3 years ago

Could you please try running it again with the -v flag and post your output?

WiiuVcExtractor -v WUP-JCGE.rpx
Lv36Blastoise commented 3 years ago 
C:\Users\Anthony.000>cd C:\Users\Anthony.000\Downloads\Downloads\zHB\wiiu-vc-extractor-0.9.0

C:\Users\Anthony.000\Downloads\Downloads\zHB\wiiu-vc-extractor-0.9.0>WiiuVcExtractor -v WUP-JCGE.rpx
Verbose output mode is set
Source extract file is C:\Users\Anthony.000\Downloads\Downloads\zHB\wiiu-vc-extractor-0.9.0\WUP-JCGE.rpx
============================================================================
Starting extraction of rom from WUP-JCGE.rpx...
============================================================================
RPX file detected!
Decompressing RPX file...
RpxHeader:
identity: 7F-45-4C-46-01-02-01-CA-FE-00-00-00-00-00-00-00
type: 65025
machine: 20
version: 1
entryPoint: 0x2FE01F8
phOffset: 0x0
shOffset: 0x40
flags: 0
ehSize: 52
phEntSize: 0
phNum: 0
shEntSize: 40
shNum: 28
shStrIndex: 25
sHeaderDataElfOffset: 0x4A0

RpxSectionHeader:
name: 0
type: 0
flags: 0
address: 0x0
offset: 0x0
size: 0
link: 0
info: 0
addrAlign: 0
entSize: 0

RpxSectionHeaderSort:
index: 1
offset: 0x94BE40

RpxSectionHeader:
name: 1
type: 1
flags: 6
address: 0x2000000
offset: 0x94BE40
size: 8
link: 0
info: 0
addrAlign: 32
entSize: 0

RpxSectionHeaderSort:
index: 2
offset: 0x94BE80

RpxSectionHeader:
name: 10
type: 1
flags: 134217734
address: 0x2000020
offset: 0x94BE80
size: 3853265
link: 0
info: 0
addrAlign: 32
entSize: 0

RpxSectionHeaderSort:
index: 3
offset: 0x640

RpxSectionHeader:
name: 16
type: 1
flags: 134217731
address: 0x10000000
offset: 0x640
size: 9736556
link: 0
info: 0
addrAlign: 4096
entSize: 0

RpxSectionHeaderSort:
index: 4
offset: 0x9497C0

RpxSectionHeader:
name: 24
type: 1
flags: 134217731
address: 0x10D48280
offset: 0x9497C0
size: 4288
link: 0
info: 0
addrAlign: 32
entSize: 0

RpxSectionHeaderSort:
index: 5
offset: 0x94A880

RpxSectionHeader:
name: 30
type: 1
flags: 134217731
address: 0x10D4CB40
offset: 0x94A880
size: 78
link: 0
info: 0
addrAlign: 32
entSize: 0

RpxSectionHeader:
name: 41
type: 8
flags: 3
address: 0x10D4CC00
offset: 0x0
size: 22225604
link: 0
info: 0
addrAlign: 256
entSize: 0

RpxSectionHeaderSort:
index: 7
offset: 0xCF8A80

RpxSectionHeader:
name: 46
type: 4
flags: 134217728
address: 0x0
offset: 0xCF8A80
size: 40857
link: 23
info: 3
addrAlign: 4
entSize: 12

RpxSectionHeaderSort:
index: 8
offset: 0xD02A40

RpxSectionHeader:
name: 59
type: 4
flags: 134217728
address: 0x0
offset: 0xD02A40
size: 309019
link: 23
info: 2
addrAlign: 4
entSize: 12

RpxSectionHeaderSort:
index: 9
offset: 0xD4E180

RpxSectionHeader:
name: 70
type: 4
flags: 134217728
address: 0x0
offset: 0xD4E180
size: 2869
link: 23
info: 4
addrAlign: 4
entSize: 12

RpxSectionHeaderSort:
index: 10
offset: 0x94A900

RpxSectionHeader:
name: 81
type: 2147483650
flags: 134217734
address: 0xC0003040
offset: 0x94A900
size: 37
link: 0
info: 0
addrAlign: 4
entSize: 0

RpxSectionHeaderSort:
index: 11
offset: 0x94A940

RpxSectionHeader:
name: 97
type: 2147483650
flags: 134217730
address: 0xC0003580
offset: 0x94A940
size: 29
link: 0
info: 0
addrAlign: 4
entSize: 0

RpxSectionHeaderSort:
index: 12
offset: 0x94A980

RpxSectionHeader:
name: 113
type: 2147483650
flags: 134217734
address: 0xC00035C0
offset: 0x94A980
size: 32
link: 0
info: 0
addrAlign: 4
entSize: 0

RpxSectionHeaderSort:
index: 13
offset: 0x94A9C0

RpxSectionHeader:
name: 129
type: 2147483650
flags: 134217734
address: 0xC00038C0
offset: 0x94A9C0
size: 31
link: 0
info: 0
addrAlign: 4
entSize: 0

RpxSectionHeaderSort:
index: 14
offset: 0x94AA00

RpxSectionHeader:
name: 146
type: 2147483650
flags: 134217734
address: 0xC0003A00
offset: 0x94AA00
size: 43
link: 0
info: 0
addrAlign: 4
entSize: 0

RpxSectionHeaderSort:
index: 15
offset: 0x94AA40

RpxSectionHeader:
name: 159
type: 2147483650
flags: 134217734
address: 0xC00044C0
offset: 0x94AA40
size: 44
link: 0
info: 0
addrAlign: 4
entSize: 0

RpxSectionHeaderSort:
index: 16
offset: 0x94AA80

RpxSectionHeader:
name: 177
type: 2147483650
flags: 134217734
address: 0xC0004D40
offset: 0x94AA80
size: 32
link: 0
info: 0
addrAlign: 4
entSize: 0

RpxSectionHeaderSort:
index: 17
offset: 0x94AAC0

RpxSectionHeader:
name: 194
type: 2147483650
flags: 134217734
address: 0xC0004F00
offset: 0x94AAC0
size: 30
link: 0
info: 0
addrAlign: 4
entSize: 0

RpxSectionHeaderSort:
index: 18
offset: 0x94AB00

RpxSectionHeader:
name: 208
type: 2147483650
flags: 134217734
address: 0xC0005180
offset: 0x94AB00
size: 30
link: 0
info: 0
addrAlign: 4
entSize: 0

RpxSectionHeaderSort:
index: 19
offset: 0x94AB40

RpxSectionHeader:
name: 225
type: 2147483650
flags: 134217734
address: 0xC0005240
offset: 0x94AB40
size: 44
link: 0
info: 0
addrAlign: 4
entSize: 0

RpxSectionHeaderSort:
index: 20
offset: 0x94AB80

RpxSectionHeader:
name: 243
type: 2147483650
flags: 134217734
address: 0xC0005B00
offset: 0x94AB80
size: 58
link: 0
info: 0
addrAlign: 4
entSize: 0

RpxSectionHeaderSort:
index: 21
offset: 0x94ABC0

RpxSectionHeader:
name: 261
type: 2147483650
flags: 134217730
address: 0xC0007D80
offset: 0x94ABC0
size: 31
link: 0
info: 0
addrAlign: 4
entSize: 0

RpxSectionHeaderSort:
index: 22
offset: 0x94AC00

RpxSectionHeader:
name: 279
type: 2147483650
flags: 134217734
address: 0xC0007E40
offset: 0x94AC00
size: 49
link: 0
info: 0
addrAlign: 4
entSize: 0

RpxSectionHeaderSort:
index: 23
offset: 0x94AC40

RpxSectionHeader:
name: 297
type: 2
flags: 134217730
address: 0xC0000000
offset: 0x94AC40
size: 1988
link: 24
info: 19427
addrAlign: 4
entSize: 16

RpxSectionHeaderSort:
index: 24
offset: 0x94B440

RpxSectionHeader:
name: 305
type: 3
flags: 134217730
address: 0xC0001690
offset: 0x94B440
size: 2318
link: 0
info: 0
addrAlign: 1
entSize: 0

RpxSectionHeaderSort:
index: 25
offset: 0x94BD80

RpxSectionHeader:
name: 313
type: 3
flags: 134217730
address: 0xC0002ED2
offset: 0x94BD80
size: 153
link: 0
info: 0
addrAlign: 1
entSize: 0

RpxSectionHeaderSort:
index: 26
offset: 0x4A0

RpxSectionHeader:
name: 0
type: 2147483651
flags: 0
address: 0x0
offset: 0x4A0
size: 112
link: 0
info: 0
addrAlign: 4
entSize: 4

RpxSectionHeaderSort:
index: 27
offset: 0x540

RpxSectionHeader:
name: 0
type: 2147483652
flags: 0
address: 0x0
offset: 0x540
size: 224
link: 0
info: 0
addrAlign: 4
entSize: 0

RpxSectionHeaderSort:
index: 26
offset: 0x4A0

RpxSectionHeaderSort:
index: 27
offset: 0x540

RpxSectionHeaderSort:
index: 3
offset: 0x640

RpxSectionHeaderSort:
index: 4
offset: 0x9497C0

RpxSectionHeaderSort:
index: 5
offset: 0x94A880

RpxSectionHeaderSort:
index: 10
offset: 0x94A900

RpxSectionHeaderSort:
index: 11
offset: 0x94A940

RpxSectionHeaderSort:
index: 12
offset: 0x94A980

RpxSectionHeaderSort:
index: 13
offset: 0x94A9C0

RpxSectionHeaderSort:
index: 14
offset: 0x94AA00

RpxSectionHeaderSort:
index: 15
offset: 0x94AA40

RpxSectionHeaderSort:
index: 16
offset: 0x94AA80

RpxSectionHeaderSort:
index: 17
offset: 0x94AAC0

RpxSectionHeaderSort:
index: 18
offset: 0x94AB00

RpxSectionHeaderSort:
index: 19
offset: 0x94AB40

RpxSectionHeaderSort:
index: 20
offset: 0x94AB80

RpxSectionHeaderSort:
index: 21
offset: 0x94ABC0

RpxSectionHeaderSort:
index: 22
offset: 0x94AC00

RpxSectionHeaderSort:
index: 23
offset: 0x94AC40

RpxSectionHeaderSort:
index: 24
offset: 0x94B440

RpxSectionHeaderSort:
index: 25
offset: 0x94BD80

RpxSectionHeaderSort:
index: 1
offset: 0x94BE40

RpxSectionHeaderSort:
index: 2
offset: 0x94BE80

RpxSectionHeaderSort:
index: 7
offset: 0xCF8A80

RpxSectionHeaderSort:
index: 8
offset: 0xD02A40

RpxSectionHeaderSort:
index: 9
offset: 0xD4E180

Decompression complete.
Checking if this is an NES VC title...
Checking WUP-JCGE.rpx.extract...
Not an NES VC Title
Checking if this is an SNES VC title...
Checking WUP-JCGE.rpx.extract...
Checking for the SNES WUP header
Determining SNES Rom header type (LoROM or HiROM)
Seeking to 0x8650 to check for possible LoROM header.
Seeking to 0x10650 to check for possible HiROM header.
Checking potential header: 53-54-52-45-45-54-20-46-49-47-48-54-45-52-20-41-4C-50-48-41-32-32-43-0C-00-01-33-00-65-24-9A-DB
Checking for valid SNES header title
Title: STREET FIGHTER ALPHA2
Ensure header rom size 0xC is valid.
Ensure header checksum 0xDB9A is valid.
Checking potential header: 53-54-52-45-45-54-20-46-49-47-48-54-45-52-20-41-4C-50-48-41-32-32-43-0C-00-01-33-00-66-24-99-DB
Checking for valid SNES header title
Title: STREET FIGHTER ALPHA2
Ensure header rom size 0xC is valid.
Ensure header checksum 0xDB99 is valid.
Could not determine header type since both HiROM and LoROM headers were valid.
SNES header type is Unknown
Not an SNES VC Title
Checking if this is a Famicom Disk System VC title...
Checking WUP-JCGE.rpx.extract...
Not a FDS VC Title
============================================================================
FAILURE: Could not successfully identify the rom type for WUP-JCGE.rpx
wheatevo commented 3 years ago

Looks like the case where both headers are valid needs to be handled in the code (at https://github.com/wheatevo/wiiu-vc-extractor/blob/master/WiiuVcExtractor/RomExtractors/SnesVcExtractor.cs#L281). We should be able to do that by checking the reset vector (0xFFFC) to determine the opcode and using that information to set the appropriate header type.

Header check:

Determining SNES Rom header type (LoROM or HiROM)
Seeking to 0x8650 to check for possible LoROM header.
Seeking to 0x10650 to check for possible HiROM header.
Checking potential header: 53-54-52-45-45-54-20-46-49-47-48-54-45-52-20-41-4C-50-48-41-32-32-43-0C-00-01-33-00-65-24-9A-DB
Checking for valid SNES header title
Title: STREET FIGHTER ALPHA2
Ensure header rom size 0xC is valid.
Ensure header checksum 0xDB9A is valid.
Checking potential header: 53-54-52-45-45-54-20-46-49-47-48-54-45-52-20-41-4C-50-48-41-32-32-43-0C-00-01-33-00-66-24-99-DB
Checking for valid SNES header title
Title: STREET FIGHTER ALPHA2
Ensure header rom size 0xC is valid.
Ensure header checksum 0xDB99 is valid.
Could not determine header type since both HiROM and LoROM headers were valid.
Lv36Blastoise commented 3 years ago

How would I go about the next step? sorry kinda new at this

wheatevo commented 3 years ago

The next step will require a code change to the SnesVcExtractor to better handle this situation. If you would like feel free to submit a pull request with a correction, otherwise I should be able to take a look at this later this week/this weekend.

Lv36Blastoise commented 3 years ago

how do i make a pull request again?

wheatevo commented 3 years ago

After some more investigation since this is an S-DD1 game simply extracting it using our existing logic to force the header type to HiROM or LoROM will result in a broken dump with scrambled visuals.

This reddit post details some of the differences between the SNES original and the VC release: https://www.reddit.com/r/miniSNES/comments/75ffcj/street_fighter_alpha_2_support_in_canoe_is/

From looking at the data, it seems like it is similarly done to how the PCM data was packed. The SDD1 (53 44 44 31) signature precedes each SDD1 data pointer.

More header info: https://gist.github.com/anpage/c1085055db0242ea3c7558dab56712a5

EDIT: Implemented the S-DD1 compressor and determined the format of the decompressed S-DD1 data. Seems like 0x18 in the VC header points to the S-DD1 data offset but the first 16 bytes of that data is another header indicating the start of the S-DD1 data:

S-DD1 Data Format

Start Size Description Notes
0x00 0x4 Location of S-DD1 data offset S-DD1 section offset + 0x10
0x04 0xC Zero byte padding (00 00 00 00 00 00 00 00 00 00 00 00)
0x10 until end of file Decompressed S-DD1 data

Going to implement this and retest to determine if this works as anticipated. As a bonus, we only need to rewrite the SDD1 header and pointer in the rom (8 bytes) instead of the entire section since only the first 8 bytes of each piece of compressed graphics data is overwritten by the SDD1 decompressed graphics pointer.

It ended up that replacing only the SDD1 header and pointer was insufficient and the compressed data needed to be written back in its entirety. It's possible this may indicate a difference in the way we are compressing the data vs. the original image, but it is successfully decompressed and the extracted data appears to work.

Initial Extraction by Forcing Rom Header Type

This is the output of the title screen received when simply forcing the header to be accepted as LoROM: image

S-DD1 Replacement of S-DD1 Pointers Only

This is the output of the title screen received when only replacing the S-DD1 header and pointer (8 bytes) with the recompressed data: image

S-DD1 Replacement with Complete Recompressed Data

This is the output of the title screen received when only replacing all S-DD1 data with the recompressed data: image

wheatevo commented 3 years ago

This is resolved as of release 1.0.0:

Rom uses S-DD1, recompressing S-DD1 data...
Extracting S-DD1 Data...
Appended rom data size: 6778754
2815 S-DD1 pointers found
Reading S-DD1 data into memory...
Replaced all S-DD1 pointers with compressed S-DD1 data
Writing to Street Fighter Alpha 2.sfc...
Writing SNES rom data...
SNES rom has been created successfully at Street Fighter Alpha 2.sfc
============================================================================
WUP-JCGE.rpx has been extracted to Street Fighter Alpha 2.sfc successfully.
============================================================================

Note that the current compression algorithm implementation is pretty inefficient and takes quite awhile, we may be able to improve performance in the future but since this will only apply to one game it may not be worth the effort.