PRX tool can list and extract files from "First Dance", with some bugs

[AlanisSmithee]

The biggest being that the List command may be showing duplicates of various files and some files are just not extracted. CHOIC2.PRX is a good example.

$ ./prx -t ./dance/RESOURCE/CHOIC2.PRX
>> 37 entries:
1     SCENE_2_2_BG      XPK   1200    217802
2     LEFT_HEAD         XPK   1001    18117
3     RIGHT_BALLOON_H   XPK   1030    1690
4     MC_2_2_NO         XPK   2220    7441
5     MC_2_1_POSITIVE   XPK   2110    6838
6     UP_HEAD           XPK   1002    18758
7     LEFT_BALLOON      XPK   1011    1453
8     LEFT_BALLOON      XPK   1012    1515
9     LEFT_BALLOON      XPK   1013    1546
10    RELIEVED          Aif   2320    193054
11    NEGATIVE          Aif   2120    336202
12    UNSURE            Aif   2230    203902
13    FEISTY            Aif   2330    245962
14    MC_2_1_NEGATIVE   XPK   2120    7168
15    SCENE_2_3_BG      XPK   1300    198149
16    MC_2_1_INDIFFER   XPK   2130    7367
17    MC_2_2_UNSURE     XPK   2230    6881
18    MC_2_2_YES        XPK   2210    6711
19    CENTER_BALLOON    XPK   1021    1408
20    CENTER_BALLOON    XPK   1022    1753
21    CENTER_BALLOON    XPK   1023    1574
22    LEFT_BALLOON_HI   XPK   1010    1463
23    MC_2_3_DISAPPOI   XPK   2310    6881
24    CENTER_BALLOON_   XPK   1020    1396
25    INDIFFERENT       Aif   2130    345928
26    DISAPPOINTED      Aif   2310    286442
27    MC_2_3_RELIEVE    XPK   2320    7113
28    SCENE_2_1_BG      XPK   1100    254679
29    CENTER_HEAD       XPK   1000    19125
30    RIGHT_BALLOON     XPK   1031    1719
31    RIGHT_BALLOON     XPK   1032    1662
32    RIGHT_BALLOON     XPK   1033    1534
33    RIGHT_HEAD        XPK   1003    18493
34    MC_2_3_FIESTY     XPK   2330    7292
35    POSITIVE          Aif   2110    204850
36    YES               Aif   2210    227300
37    NO                Aif   2220    161180

Here's what was extracted. 30 files vs 37.

IDGLASS.PRX is even funnier.

$ ./prx -t ./dance/RESOURCE/IDL/IDGLASS.PRX
>> 18 entries:
1     DFTG_GENERICFIN   Aif   32027   162906
2     DFTG_GENERICFIN   Aif   32024   193874
3     DFTG_GENERICFIN   Aif   32033   269270
4     DFTG_GENERICFIN   Aif   32021   446634
5     DFTG_GENERICFIN   Aif   32030   282030
6     TG_ANIMATED       XPK   20705   1359874
7     TG_ANIMATED       XMV   20705   1416
8     DFTG_GENERICFIN   Aif   32029   162906
9     DFTG_GENERICFIN   Aif   32026   429764
10    DFTG_GENERICFIN   Aif   32035   128146
11    DFTG_GENERICFIN   Aif   32023   189784
12    DFTG_GENERICFIN   Aif   32032   345446
13    DFTG_GENERICFIN   Aif   32028   242698
14    DFTG_GENERICFIN   Aif   32025   164122
15    DFTG_GENERICFIN   Aif   32034   239810
16    DFTG_GENERICFIN   Aif   32022   278840
17    DFTG_GENERICFIN   Aif   32031   192110
18    TG_ANIMATED       Aif   20705   477654

Some of the character archives also have the "phantom member" problem that shows up in some of "Secret Invitation"'s archives. I'll put those in that thread.

[dressupgeekout]

Yeah, you're right, there are definitely duplicates. Bleh.

7     LEFT_BALLOON      XPK   1011    1453
8     LEFT_BALLOON      XPK   1012    1515
9     LEFT_BALLOON      XPK   1013    1546

This could be indicative of several things:

• Maybe there's a bug in the PRX tool which doesn't interpret the full name of an archive member.
• Maybe First Dance's PRX format is different, perhaps one which allows for longer filenames than the other games. I'll need original First Dance data to confirm this, but I don't have First Dance.
• Maybe we actually have to append the "resource ID" (the second-to-last column) to the the file names. So we'll be extracting LEFT_BALLOON-1011.XPK, LEFT_BALLOON-1012.XPK, etc.

[AlanisSmithee]

Tried listing and extracting stuff again with build 1e1e44a (I think). Stuff seems to be listing properly now and I didn't get any errors when I extracted one of the known problem archives. But it seems like the PRX tool is not extracting the last file in any given archive. This looks like it might be the case with Tricky and Invitation as well.

Edit: MIKO.PRX from Dance appears to be missing 15(!) files when extracted. Weird.

Edit2: SHAR.PRX from Dance is missing 12. Oof. That's no good.

[dressupgeekout]

OK, there might be a few different things going on.

(1)

But it seems like the PRX tool is not extracting the last file in any given archive.

I did touch the PRX tool in a commit later than 1e1e44a -- Hopefully I didn't introduce any regressions with https://github.com/dressupgeekout/lunapurpura/commit/ce4b0d9117698cdfdd239625aa584c3967fb30ed -- I'll double check, regardless

(2) @AlanisSmithee are you able to obtain the "missing" files if you extract them individually? I don't have Dance files but I'm 99% certain they're only "missing" when you extract everything at once (prx -x -a) because, e.g. it thinks it has to extract LEFT_BALLOON.XPK and then it will extract another LEFT_BALLOON.XPK but the first one gets overwritten.

[AlanisSmithee]

In build 1e1e44a, yes. The missing files can be extracted individually. I haven't compiled any of the later builds yet. Two of the files I know of that occur at the end of an archive and don't get extracted (TERMINAL_AMB.aif in AMBIENT and ROMANTIC.aif in CHOIC1_1) are uniquely named files though.

[dressupgeekout]

OK, I can confirm that the last PRX entry is not getting extracted with prx -x -a, even with the latest code in 'master' as of this writing. Must be an off-by-one error somewhere, but I thought I had already fixed this...

[dressupgeekout]

Lol, that was silly. The "can't extract last member" error should be fixed in 0bdcdf32f8944a21b476a60ccd12e51d34a92d91.

[AlanisSmithee]

Yup, that's working now. However, I am still getting list/extract result discrepancies.

$ ./prx -t ./dance/RESOURCE/IDL/MIKO.PRX
>> 175 entries:
1     MIKCLIP_01_04H    XPK   43864   85797
2     MIKLUNCH_01_H     XPK   43887   5848
3     MIKCLIP_01_RH     XPK   43859   391
4     MIKPACK_01_RH     XPK   43823   1412
5     MIKGIFTMIKO_01_   XPK   43825   3964
6     MIKBIRTHDAY_01_   XPK   43869   1893
7     MIKCSGLEFT_01_R   XPK   43809   168
8     MIKPENCIL_01_H    XPK   43882   3836
9     MIKCAMPAPP_01_0   XPK   43855   61547
10    MIK_NECKLACE      XPK   43817   20657
11    MIKIDEABK_01_01   XPK   43828   75211
12    MIK_ANIPEONY      XPK   43813   476404
13    MIKPLANT_01_H     XPK   31000   3637
14    MIKPLANT_01_RH    XPK   31001   2582
15    MIKPRB_01_02H     XPK   47493   142898
16    MIKO_N5_S3_1E     TXT   21008   1206
17    MIKLUNCH_01_02H   XPK   43890   34024
18    MIKO_N2_S2_2Y     TXT   21005   942
19    MIKTRUTH_01_H     XPK   43884   2186
20    MIKPRB_01_H       XPK   47490   3981
21    MIKO_N8_S4C_2F    TXT   21011   591
22    MIKIDEABK_01_RH   XPK   43826   387
23    MIKWALL_01_RH     XPK   43847   2501
24    MIKFOLD_01_H      XPK   43878   885
25    MIKHEARTS_01_RH   XPK   31004   4720
26    MIKBONSI_01_03H   XPK   43837   5779
27    MIK_hearts        XPK   31005   62165
28    MIKSHELF_01_H     XPK   43897   4178
29    MIK_CLOGS         XPK   31011   40726
30    MIK_CLOGS         XMV   31011   1464
31    MIKBONSI_01_02H   XPK   43836   5455
32    MIKCSG_01_RH      XPK   43804   800
33    MIKCSG_01_01H     XPK   43805   100029
34    MIKBONSI_01_01H   XPK   43835   5716
35    MIKCSGLEFT_01_H   XPK   43807   261
36    MIK_LUNCHBOX      XPK   43891   47453
37    MIKBOXNOTEWH_01   XPK   43880   3569
38    MIKANIMALBOX_01   XPK   43872   2916
39    MIKANIMALBOX      XPK   43874   7333
40    MIKBONSI_01_04R   XPK   43838   2582
41    MIKCLOGS_01_RH    XPK   31010   1114
42    MIKHOOKS_01_H     XPK   43846   504
43    MIKPACK_01_01H    XPK   43819   69837
44    MIK_BALLOON       XPK   43868   171449
45    MIK_BALLOON       XMV   43868   1208
46    MIKIDEABK_01_04   XPK   47499   106960
47    MIK_ANIMBOX       Aif   43874   77277
48    MIK_CLIMBER       Aif   31008   220582
49    MIK_BNSISF_V3     Aif   43844   91438
50    MIK_BNSISS_V2     Aif   43843   91438
51    MIK_PLANT         Aif   31002   220606
52    MIK_CSGphoto      XMV   43896   984
53    MIK_CSGPHOTO      Aif   43896   220606
54    MIKWATCH_01_H     XPK   43814   1429
55    MIKPACK_01_04H    XPK   43822   68021
56    MIKLUNCH_01_01H   XPK   43889   31667
57    MIKCLIP_01_H      XPK   43858   7311
58    MIKPACK_01_H      XPK   43818   11203
59    MIKPRBRIGHT_01_   XPK   47497   907
60    MIKPOSTER_01_H    XPK   43895   21106
61    MIKPACK_01_03H    XPK   43821   83183
62    MIKNECKLACE_01_   XPK   43801   1650
63    MIKCAMPAPP_01_H   XPK   43852   1111
64    MIKBALLOON_01_H   XPK   43866   409
65    MIKPACK_01_02H    XPK   43820   67105
66    MIK_ANIBNSIWS     XPK   43842   304031
67    MIK_ANIBNSISS     XPK   43843   218081
68    MIK_ANIBNSIWS     XMV   43842   1176
69    MIK_ANIBNSISS     XMV   43843   1176
70    MIK_ANIBNSIFW     XPK   43845   261549
71    MIK_ANIBNSIFW     XMV   43845   1176
72    MIKPENCIL_01_RH   XPK   43883   1304
73    MIKTRUTH_01_RH    XPK   43885   2451
74    MIKWATCH_01_RH    XPK   43815   479
75    MIKVALENTINE_01   XPK   31013   3828
76    MIKANIMALBOX_01   XPK   43873   640
77    MIKIDEABK_01_03   XPK   47498   110793
78    MIKO_N6_S3_1E     TXT   21009   682
79    MIKBIRTHDAY_01_   XPK   43870   2175
80    MIKO_N4_S2_3D     TXT   21007   1001
81    MIKO_N3_S2_2Y     TXT   21006   840
82    MIKO_J3_S2_3D     TXT   21002   475
83    miko_n1_s1_1f     TXT   21004   587
84    MIKO_J2_S2_2Y     TXT   21001   637
85    MIKNEC_01_H       XPK   43848   1163
86    MIKPEONY_01_H     XPK   43811   14907
87    MIKO_J4_S4C_2F    TXT   21003   1113
88    WP_REPORTCARD_0   XPK   43877   2179
89    WP_REPORTCARD_0   XPK   43876   1955
90    miko_cb1_s3_1e    TXT   21012   163
91    MIKBONSI_01_03R   XPK   43841   2112
92    MIKCSGPHOTO_01_   XPK   43899   12627
93    MIKWALL_01_H      XPK   43800   16273
94    MIKCSG_01_02H     XPK   43806   86312
95    MIKCSG_01_H       XPK   43803   13586
96    MIKNEC_01_02H     XPK   43851   48766
97    NAKNEC_01_01H     XPK   43850   40478
98    MIKIDEABK_01_05   XPK   31012   172374
99    MIKPEONY_01_RH    XPK   43812   634
100   MIK_CSGphoto      XPK   43896   440071
101   MIKPRBLEFT_01_H   XPK   47494   1228
102   MIKBONSI_01_02R   XPK   43840   1494
103   MIKNECKLACE_01_   XPK   43802   1094
104   MIKIDEABKRIGHT_   XPK   43829   255
105   MIKCLIP_01_05H    XPK   43865   89725
106   TRUTH_01_02H      XPK   43893   40499
107   D4LM_MIK6         Aif   45236   353219
108   D4LM_MIK5         Aif   45235   285738
109   D4LM_MIK4         Aif   45234   361774
110   D4LM_MIK3         Aif   45233   198714
111   D4LM_MIK2         Aif   45232   214250
112   D4LM_MIK1         Aif   45231   332026
113   MIK_BNSIWS_V1     Aif   43842   91438
114   MIK_PEONY_V4      Aif   43813   311846
115   MIK_CLOGS         Aif   31011   220582
116   MIK_WATCH_V4      Aif   43816   214826
117   MIK_CARD          XMV   43871   1080
118   MIK_CARD          Aif   43871   75072
119   MIK_ANIWATCH      XMV   43816   2520
120   MIK_BALLOON       Aif   43868   110352
121   MIKIDEABKRIGHT_   XPK   43830   168
122   MIKCLIP_01_04H    XPK   43863   84727
123   MIKCLIP_01_03H    XPK   43862   81622
124   MIKLUNCH_01_RH    XPK   43888   783
125   MIKCLIP_01_02H    XPK   43861   76271
126   MIKCLIP_01_01H    XPK   43860   67220
127   MIKO_TG33_S3_1E   TXT   21014   119
128   MIK_ANIBNSISF     XPK   43844   218929
129   MIK_ANIBNSISF     XMV   43844   1176
130   MIKPRBLEFT_01_R   XPK   47495   957
131   MIKPENCIL_01_H    XPK   43881   3558
132   MIK_CLIMBER       XPK   31008   589709
133   MIK_CLIMBER       XMV   31008   1784
134   MIKANIMALBOX_01   XPK   43875   3481
135   miko_cb2_s4a_2f   TXT   21013   101
136   MIKPRB_01_RH      XPK   47491   406
137   MIKCLOGS_01_H     XPK   31009   2376
138   MIKPRB_01_01H     XPK   47492   61652
139   MIKPRBRIGHT_01_   XPK   47496   1124
140   MIKCLIMBER_01_H   XPK   31006   20009
141   MIKO_J1_S1_1F     TXT   21000   735
142   MIKCAMPAPP_01_0   XPK   43854   48003
143   MIKIDEABK_01_00   XPK   43827   64044
144   MIKFOLD_01_RH     XPK   43879   418
145   MIKO_N7_S4A_2F    TXT   21010   1470
146   MIKBONSI_01_01R   XPK   43839   1354
147   MIK_PLANT         XPK   31002   187072
148   MIK_PLANT         XMV   31002   1784
149   MIKNEC_01_RH      XPK   43849   1371
150   MIKHEARTS_01_H    XPK   31003   5442
151   MIKCLIMBER_01_R   XPK   31007   944
152   MIKCAMPAPP_01_R   XPK   43853   1308
153   MIKBALLOON_01_R   XPK   43867   516
154   MIKIDEABKLEFT_0   XPK   43833   188
155   MIKIDEABKLEFT_0   XPK   43832   335
156   MIKBONSI_01_04H   XPK   43834   3637
157   MIK_CARD          XPK   43871   12787
158   MIKVALENTINE_01   XPK   31014   553
159   MIK_ANIWATCH      XPK   43816   26535
160   MIKBOOKS_01_H     XPK   43824   5340
161   MIKCSGRIGHT_01_   XPK   43810   162
162   MIKCSGPHOTO_01_   XPK   43898   574
163   MIKCSGRIGHT_01_   XPK   43808   244
164   TRUTH_01_01H      XPK   43886   22264
165   TRUTH_01_03H      XPK   43894   39555
166   MIKIDEABK_01_02   XPK   43831   107609
167   MIK_NECKLACE      XMV   43817   1464
168   MIK_NECKLACE      Aif   43817   88302
169   MIK_ANIPEONY      XMV   43813   3576
170   MIK_BNSIFW_V5     Aif   43845   91346
171   MIK_hearts        XMV   31005   1096
172   MIK_hearts        Aif   31005   176482
173   MIK_LUNCHBOX      XMV   43891   888
174   MIK_LUNCHBOX      Aif   43891   48612
175   MIKANIMALBOX      XMV   43874   1304

Unfortunately, I can't tell exactly which files are missing, since it sorts itself into alphabetical order when extracted. It's probably what you described: stuff getting accidentally overwritten.

Edit: Going back to CHOIC2, extract all grabbed 32 files this time instead of 30.

[dressupgeekout]

Finally got a hold of Dance files from the Internet Archive. I'd prefer the actual CD-ROM, but at the very least, I'm able to reproduce this duplicate filenames problem you're seeing. The PRX format itself is not different between Dance and the other games. Peeking inside of IDGLASS.PRX, there really are multiple files called DFTG_GENERICFIN.

The PRX format can only handle file names of up to 15 characters, and that doesn't seem to be any different for Dance. I suspect that the original developers did have files with different names, but stuffing them inside of a PRX file simply truncates the filenames. That information is just... lost.

And it's not like the full filenames are strictly necessary, from the game engine's perspective. Looks like the resource ID ("RID") reigns supreme. That's a critical detail.

Anyway, it looks like the best way to fix this Dance problem is to append the RID to the filename. I had already made such a suggestion earlier, but now I think I'm going to use this pattern: <RID>-<filename>.<extension>

[dressupgeekout]

Candidate fix in https://github.com/dressupgeekout/lunapurpura/pull/44

[dressupgeekout]

Heh, there's at least one instance of the multiple files literally being duplicates, rather than merely different files with truncated filenames.

Here's WHIT.PRX from First Dance:

charlotte@dawnchorus:~/devel/lunapurpura/love/dance/IDL/WHIT$ shasum -a256 *shell*
a982f208be0897c02039cf4210cf33f56eb5622b1969c09a7e6945d604750674  31468-WHI_shellmirror.Aif
a982f208be0897c02039cf4210cf33f56eb5622b1969c09a7e6945d604750674  31469-WHI_shellmirror.Aif
a982f208be0897c02039cf4210cf33f56eb5622b1969c09a7e6945d604750674  31470-WHI_shellmirror.Aif
a982f208be0897c02039cf4210cf33f56eb5622b1969c09a7e6945d604750674  31471-WHI_shellmirror.Aif

Contrast this against the different Truth Glasses announcements:

charlotte@dawnchorus:~/devel/lunapurpura/love/dance/IDL/IDGLASS$ shasum -a256 *DFTG_GENERICFIN.Aif
6e031c7ead047198e6e2dee3f0ec82531bf69bbec3781136455f20c001b9c11b  32021-DFTG_GENERICFIN.Aif
124979712a2a93aedc4703b7dfc2a43647966b43a53f895b83d0f7682b3feace  32022-DFTG_GENERICFIN.Aif
52fdc50c01e6541dd6124501339952164e71e83233c2c7a261ca892b3b98954d  32023-DFTG_GENERICFIN.Aif
1c791fd8528e9c1137d7d37b80d3e7382263e1dc3f09c4f689f3a00b89836f12  32024-DFTG_GENERICFIN.Aif
c15b813955cfd7722610baf34101da8a8954dea7e94cb574893c004c15d172ce  32025-DFTG_GENERICFIN.Aif
267fa089556d0268d7897b17440ecfb098e981581cd65e4395db8f14b4c7f496  32026-DFTG_GENERICFIN.Aif
0eefcf82608d6aee08356e8a97fbbcc6f01862ab2f9fc2639e7086aef766b596  32027-DFTG_GENERICFIN.Aif
b056a92bcf0d5f7b9c1e6ebb67f90b6cea5b646df3583aff2fdc7e8e4aa82609  32028-DFTG_GENERICFIN.Aif
2bcb3b1be3c15d6f91bccda7d4bf7361c7fe9feefe217ba22c4de09d82986702  32029-DFTG_GENERICFIN.Aif
c4900c52afbb21be2e912437a1a5db219ccdbef5a7467401c544b060b1fbe482  32030-DFTG_GENERICFIN.Aif
25c3b4e4dce026e0d930d2a37c6e34d01757ef6f49f856f62ea15e7b687a1bcd  32031-DFTG_GENERICFIN.Aif
7fc36faaa0afea895a3ca0098b5c5e8c47d78ce7ceb5962da9272ab5b5881b1f  32032-DFTG_GENERICFIN.Aif
19bf6f5b60bae7ddbbd488511e30c536f5ed793804e51ccf6303dbaebacbbf01  32033-DFTG_GENERICFIN.Aif
b019b77b6b51fdea03374f427e2d149417b1f05f4e2d4ebbbc27fe42febd46fa  32034-DFTG_GENERICFIN.Aif
aa11c1c180265baaab5cf587e9a867fc9f300d7f910add1186afb72a6a5f7134  32035-DFTG_GENERICFIN.Aif

So, well, that's interesting and annoying.

[dressupgeekout]

https://github.com/dressupgeekout/lunapurpura/pull/44 has been merged in. 'master' @ 6e1528f71400b74032bb6fdcb31d92f38f2f67c1 or later should fix things

[AlanisSmithee]

That worked. 175 files listed in Miko.prx. 175 files extracted.

Adding the resource ID to the file name won't cause too much of a headache when coding the actual interpreter, will it?

[dressupgeekout]

It will have zero impact on the actual interpreter, since I've learned that the resource ID/filetype combo is what one really needs in order to uniquely identify a given asset. And besides, the only files per se in the original Rockett games are PRXes. That is, #44 doesn't affect how individual archive members are located, it only affects how they're presented.