Bpkdecryptor

[DPGrev]

First I would like to say, great work so far.

I have however problems using the bpkdecryptor. It seems that the gif files it generates cannot be opened by any image processor. They are corrupt. Is there a specific reason for this?

[enriquesomolinos]

At this moment the bpk decryptor only works with small images like face01.bpk, but color are not well showed.

In few days I want to write a little article with all file formats in my blog, and if I have enought time I can fix this utility because in the main project I decrypt the bpk files well.

Thanks for your interest :)

[DPGrev]

Thank you very much, I will be awaiting your article.

[1Simo1]

Hi Enrique! Good work!

Quote from your blog and your mod : "Driver face image can be repeated because at this moment we can’t change the faces to substitute with the real faces."

Maybe yes, they could be changed. I'm testing your bpk file algorithm from tools (I'm looking at the BPK algorithm for track BPAs), but in addition to the wrong palette, I can only decrypt part of the image (the first GIF block I think). However, in DreeRally the images are seen well, so you should have the updated and correct algorithm (in your dr.c source seems a LZW/GIF decompression with modified palette). The idea would be to extract BPK from the BPA, convert it into a gif image, modify it, reconvert it into BPK and store it again in the BPA, keeping the size of the entire BPA file unchanged.

[1Simo1]

Now after several tests, I can decipher the whole image but I also have problems with colors: the color that should be in a certain pixel is in the palette (and this is a good sign :)) but it seems that the order in which colors should be taken not the same as they appear in the PAL file.

I attach a file with Sam Speed as example

FACE01.BPK.zip

[enriquesomolinos]

Yes, you are correct. Months ago I found this problem but its not easy for me to solve. I'm happy to know that you are investigating this things :)

[1Simo1]

Almost!

Maybe there is only a value approximation in conversion from 6-bit VGA palette to 8 bit color!

Maybe trying with gif "recoding" instead of bmp gives the desired result? I'll let you know Enrique :)

almost.zip

[1Simo1]

(First part of the) Problem solved!

ok.zip

[enriquesomolinos]

ohhh!,

Great job, it could be interesting for making a image editor :) .

[1Simo1]

Thank you! :)

Another example in attachment. For now, I'm trying with BMP output yet. So, some image decodes less pixels than the total of width by height (specially intro images) and for others for now I have to guess correct width and height.

Also, the main images I'm looking at, those of the tracks, seem, as is reasonable to think, to have more than 640x480 pixels (and I mean more than double!)

CURSOR.zip

From the image in attachment, seems, and this is perfectly reasonable, that there are, at least for "animations", images composed by its sprites.

Maybe, tracks follow same logic and dat and inf indicate which sprites of the possible grid take in sequence?

[enriquesomolinos]

great updates!

At this moment i'm modifying the executable in order to change racelaps, but it coulb be interesting allow us to load resources from a directory instead of bpafiles to test the circuits modification.

[1Simo1]

Hi Enrique!

Surprise in attachment ! :D

The_Arena.zip

(This is "IMA" file decoded with his palette)

[enriquesomolinos]

can you explain how you extract this image? wich files are involved?

great job!

[1Simo1]

Thank you again!

I'm doing all these tests writing a new "complete" editor (I mean in some aspects) for the game, so of course I can :)

The bpk files are practically the data block of a gif image, with some modifications to the original lzw / gif algorithm. In many cases, as you already know better than me for months :) the palette is in a separate file, while in the images of the circuits, in those that I extracted, are in the same file. Decoding / decompression leads to a "file" with one byte per pixel, each value corresponds to the index to be read from the palette. Further variations of the game compared to a normal GIF: palette from 6-bit VGA to 8-bit GIF and further conversion of output byte as in your code.

In summary, for many images: PAL file + BPK file

For the images so far extracted from the TR files of the tracks (ie the ones I'm sure of): BPK only

I also underline again that in the tracks everything is in the BPK file(s), including a header of a few bytes followed by the 768 bytes of the PAL file which, unlike the other images taken out so far, is initially encoded and you have to "split" header from gif final data. In other words, in case of tracks, bpk decoded is a gif data block with previous header (maybe with image dimensions? I don't know yet)

All files in attachment!

The_Arena_so_far.zip

[enriquesomolinos]

a new editor? mmm i'm very expectating about that.

by my way im creating a trainer editor ti change race laps and the final circuit with the adversary. also im investigating /decompiling the circuit logic. probably in few days i write a few post about it with you and my investigations

[1Simo1]

Hi!

Excellent work with marathon on game!

Meanwhile, news in attachment!

Even if for now not completely, I changed an image in game!

capture.zip

[1Simo1]

Update : after several (and I mean several) tries, I can get the image attached, it would seem very close to the right colors for the modified image!

another_capture.zip

UPDATE! It's only one image,but ... IT WORKS! :D

ok.zip

[enriquesomolinos]

good work! things like that waste a lot of time, its a very difficult task what are you doing, so, the only thing i can say it that your are doing a excellent work.

if you need something about how game uses palletes, tell me.

by the other hand , im modifing the exe with a launcher, and at this moment i can give the adversary a vagabond! :) and four drivers can participate in the last race for example.

[1Simo1]

Thank you very much!

Without patching the exe, it's also more difficult because as you know better than me, exe controls total file size for BPAs (it's patchable of course in some bytes of exe).

All this task is to try to modify circuits, because, if I'm not mistaken also track logic relies partly on color parsing.

What I know or maybe know about tracks:

• What you see in race is part of the image extract from a bpk , magnified.

• Each Track BPA file but The Arena TR0 file, contains one image and one flip palette. As far as I know, to obtain "reverse" circuit, you have to rotate image IMA by 180 degrees, then apply flip palette. (IMA files have also a 10 byte header, 4 of which are width and height of the image, others 6 I don't know :) then palette of image coded also in BPK with whole file)

• Others images maybe are checkpoints/guide for AI and MAS almost certainly is a color mask for road/out of bounds/turbo.

All this leaves us with sce and sha files as images (?) and more of all with tab,dat and bin files. The latter have fixed size.

[enriquesomolinos]

as a I see in the decompiled exe the first 10 bytes of the ima files are not read. the program start to read the ima file at the byte 10 and read the palette of the image.

ima, lr1 sizes are read from the inf file as far as i know

[enriquesomolinos]

the ima image is the background of the circuit, the image that have few rectangular blocks represents the sectors of the circuit to avoid using shortcuts and know where you are in the lap.

the file with black border represent the positions wich the car can be in.

[1Simo1]

Thank you for your informations. I'm still watching at track data files and images combined.

[enriquesomolinos]

Hi Simone,

¿Can you extract the files of another circuit like The_Arena_so_far.zip file you extracted days ago?

I'm debugging the original exe and this information can help me (and you of course) to know the format of the tab files.

Thank you very much.

[1Simo1]

Hi Enrique,

Yes, I can extract (and I extracted) files from all circuits.

Tab files, dat files and inf file are extracted from BPA "as they are", six files in total. This leaves us with another seven (six for The Arena) BPK files to examine. As you know, they are images with an header followed by palette followed by image data.

(sce and sha files : I haven't depth these files yet)

Some remarks: tab files are always of 256 bytes, drv dat files always 2048 bytes, inf bin always 508 bytes and ohi dat always 1024 bytes. Also, mas and ima files for each track are always the same size, same for lr1 and vai; these four files have 778 byte header (10 at the beginning followed by a 768 byte palette).

Let me know what information/files can be useful to you.

[enriquesomolinos]

I'm searching the relationship between the ohi.dat and vai.bpk file, so It can be very usefull to have another vai.bpk decompressed to test a few things.

I think if I solve this problem, the ohi.dat file structure, we'll can undestand the other files structure.

[enriquesomolinos]

The sce.bpk it a set of textures/images

Offset (byte)	Size (bytes)	Information
0	1	block size
1	3152 * block size	one texture
3152 * block size +1	1	second block size
3152 * block size +2	44*second block size	this texture/image is calculated only if the second block size is not equals to 0
(3152 block size) +(44secon block size) + 2	390000 for TR1/TR2 , 300000 for others	another texture/image

I hope this information is usefull for you :).

[1Simo1]

Yes, maybe is very useful. I think they are sprites for other images not extracted yet (boxes, out of bounds floors, and similar). sha files suggests instead informations about shadows (?)

In attachment, vai files part into images and maybe palettes extracted from 768 bytes out of 778 of header file of ima, mas,vai and lr1 files.

vai_pal.zip

Moreover, watching your dr.c code, sometimes strangely some file part(s) are not read at all.

Question, so, first part of sce file is not "encoded" ? Or we have to expand BPK as in other cases so far?

Update: Nevermind :) The answer is to expand as in other BPKs

[enriquesomolinos]

Thanks for the images!

Here is how is sha file is readed:

Offset (byte)	Size (bytes)	Information
0	4	first header
4	4	second header
8	4*first header
4*first header + 8	4*first header
8*first header + 8	4*first header
12*first header + 8	4*second header
4(first header+second header+ 2first header) + 8	4*second header
12 first header + 8second header + 8	4*second header

All this elements must be decrypter.

[1Simo1]

Hi Enrique, you are welcome!

As an example , about sce bpk file for The Arena : first block size byte value should be 84, second block size byte value 24, final part of file (actually almost half of it) has zeroes.

For TR1, they should be 61 and 37 respectively.

But, for first image/texture I haven't any clue about width and height, looking at inf bin file doesn't help me, unless I have to consider multiple "sub-textures" included in first block of 3152*84 bytes

[1Simo1]

Hi Enrique!

Here is a summary table for sha / sce files. I'm pretty sure the values are accurate, but I have no clues yet about the size and height of the images / textures from the inf file.

By any chance do you have indications in other files of these values?

Track File	SCE 1st block size byte	SCE 2nd block size byte	SHA 1st header size byte	SHA 2nd header size byte	SHA EXP size	SHA OK
TR0	84	24	242	182	5096	*
TR1	61	37	207	167	4496	*
TR2	39	19	224	200	5096	*
TR3	73	12	250	197	5372	*
TR4	44	31	136	118	3056	*
TR5	63	17	252	174	5120	*
TR6	68	10	246	220	5600	*
TR7	54	10	247	189	5240	*
TR8	56	25	192	180	4472	*
TR9	70	9	254	188	5312	*

• SHA expanded output size is verifiable with the formula 8+12[1st byte size]+12[2nd byte size], or, if you prefer, you can calculate value of one header size given other header size value and total output size.

I hope this table will be useful for your tests.

[enriquesomolinos]

Hi Simone! Great work!,

At this moment this files (including the inf file) are very confusing for me. I think this files are not clean textures, probably it contains the 3d objects shown in the race. One think I'm sure is that we are near to know how this files are used in the game.

This week I try decompile and identify some parts of the circuit loading, so its posible that I can give you more information about that.

One thing I see this weekend is that when mas file contains a black color game load data from ski and blo files. Depending on one condition the game uses the ski or the blo file.

[1Simo1]

Hi Enrique!

In fact, in most mas files there is black color, maybe in others I extracted wrong palette?

Mas file so far in attachment tr_mas_files_v1.zip

[enriquesomolinos]

Here is how is inf file is readed:

Offset (byte)	Size (bytes)	Information
0	4	ima/mas file height or width
4	4	ima/mas file height or width
8	4	vai file zones
12	4	?
16	4	?
20	4	?
24	4	?
28	4	?
32	4	?
36	4	?
40	4	?
44	4	?
48	4	?
52	4	?
56	4	?
60	4 *10	unkown array
100	32*4	unkown array

Lr1 and vai heigth and width is ima file dimensions /4

As you see the third number is the vai fiel zones :). It is a little advance.

[1Simo1]

Hi Enrique, thanks for the answer

Lr1 and vai heigth and width is ima file dimensions /4 As you see the third number is the vai fiel zones :). It is a little advance.

Yes, I agree 😄 I had noticed it extracting these files. Also, following integers should be width and height of vai/lr1 files.

Question : These informations are obtained from your code tests or are deducted from file(s) itself? In the first case, I could write here a table with first 15 integers of inf file(s)

[enriquesomolinos]

The only thing I advance is in the third number. For example if you take the TR7-inf this value is 21, and its the same as the colored znes in the TR7-vai.bpk.

The other informations I've obtained is by decompiling the code and confirming this things with the files you provided me :).

The first 15 integer (probably more than one of that are floats :( ) are separated numbers, but the arrays have one relation itself.

[1Simo1]

Here you are the complete table of first 15 integer of files!

It's worth noting that "third value" following vai file zones value is always the same in file!

Track File	value	value	value	Informations
TR0	1408	512	20	ima&mas w/ima&mas h/vai fz
TR0	427	394	72	vai(&lr1) w/vai(&lr1) h/?
TR0	394	354	72	?/?/?
TR0	367	394	72	?/?/?
TR0	337	354	72	?/?/?
TR1	960	600	20	ima&mas w/ima&mas h/vai fz
TR1	277	173	77	vai(&lr1) w/vai(&lr1) h/?
TR1	245	149	77	?/?/?
TR1	219	190	77	?/?/?
TR1	188	166	77	?/?/?
TR2	960	600	24	ima&mas w/ima&mas h/vai fz
TR2	510	381	72	vai(&lr1) w/vai(&lr1) h/?
TR2	495	342	72	?/?/?
TR2	453	381	72	?/?/?
TR2	438	342	72	?/?/?
TR3	800	956	31	ima&mas w/ima&mas h/vai fz
TR3	518	275	0	vai(&lr1) w/vai(&lr1) h/?
TR3	554	297	0	?/?/?
TR3	518	316	0	?/?/?
TR3	554	337	0	?/?/?
TR4	600	1024	17	ima&mas w/ima&mas h/vai fz
TR4	413	479	88	vai(&lr1) w/vai(&lr1) h/?
TR4	440	500	88	?/?/?
TR4	385	519	88	?/?/?
TR4	414	540	88	?/?/?
TR5	1280	456	30	ima&mas w/ima&mas h/vai fz
TR5	851	356	18	vai(&lr1) w/vai(&lr1) h/?
TR5	886	342	18	?/?/?
TR5	914	378	18	?/?/?
TR5	950	364	18	?/?/?
TR6	800	800	17	ima&mas w/ima&mas h/vai fz
TR6	610	311	48	vai(&lr1) w/vai(&lr1) h/?
TR6	644	293	48	?/?/?
TR6	628	264	48	?/?/?
TR6	662	246	48	?/?/?
TR7	1016	716	21	ima&mas w/ima&mas h/vai fz
TR7	340	577	24	vai(&lr1) w/vai(&lr1) h/?
TR7	370	610	24	?/?/?
TR7	397	577	24	?/?/?
TR7	427	610	24	?/?/?
TR8	1040	676	28	ima&mas w/ima&mas h/vai fz
TR8	576	403	72	vai(&lr1) w/vai(&lr1) h/?
TR8	525	368	72	?/?/?
TR8	472	403	72	?/?/?
TR8	426	368	72	?/?/?
TR9	864	736	22	ima&mas w/ima&mas h/vai fz
TR9	577	171	84	vai(&lr1) w/vai(&lr1) h/?
TR9	536	170	84	?/?/?
TR9	540	204	84	?/?/?
TR9	497	203	84	?/?/?

[enriquesomolinos]

The vai/lr1 dimensions are calculated via the ima and mas dimensions in the code:

circuitVaiBpk_width_4A6858 = circuitWidth_464F40>> 2; circuitVaiBpk_height_503508 = circuitHeight_4A7CF8>> 2; circuitLR1Bpk_width_4A9B90 = circuitWidth_464F40>> 2; circuitLR1Bpk_height_467000 = circuitHeight_4A7CF8>> 2;

Thanks for the table!, I see this numbers and it apears to be points in the ima/mas file, probably the 4 start race positions. The repeated value, at this moment I have no idea what is this. I need to check it with the code!

[enriquesomolinos]

The TR0 circuit is clear :)

Track File	value	value	value	Informations
TR0	1408	512	20	ima&mas w/ima&mas h/vai fz
TR0	427	394	72	driver 1 starting position w /driver 1 starting position h /?
TR0	394	354	72	driver 2 starting position w /driver 2 starting position h /?
TR0	367	394	72	driver 3 starting position w /driver 3 starting position h /?
TR0	337	354	72	driver 4 starting position w /driver 4 starting position h /?

I take this circuit because starting grid position are in horizontal position.

[1Simo1]

yes, sorry, my mistake in table :) (by the way, you are welcome)

4 10 and 32 4 array are always the same in each file? If this is the case, there are 280 bytes completly unknown (no clues about size nor about meaning)

280 = 508-60-(4 10)-(32 4)

[enriquesomolinos]

I don't explain well ,

Taken the TR0 table the first car starts the race on the pixel 4271408 + 394 (first position y axis circuit width + first position x axis) of the ima file.

[1Simo1]

Yes, and watching other files I can confirm you that they are x and y coordinates of the starting position in ima (or mas) file.

I'm wondering where are coordinates of drivers starting positions in reverse circuit case

(I'll try to rotate image and guess where starting points are)

[enriquesomolinos]

For reverse circuits this points are calculated as the image is reversed. Think that the reverse circuit is fully calculated in code, except the pal file. So its posible to play in the arena reversed :)

[1Simo1]

Yes, you're right, there aren't searched pixel values of reversed circuit.

Meanwhile, here you are table of "first unknown array" of 4 * 10 from your inf table

Track File	value	value	value	value
TR0	555	358	799	385
TR0	1130	361	1218	169
TR0	1042	152	880	132
TR0	675	143	374	133
TR0	280	146	155	206
TR0	167	315	324	392
TR0	181	176	1243	256
TR0	10	10	10	10
TR0	509	437	0	1
TR0	1123	322	1	1
TR1	390	67	543	82
TR1	645	77	919	158
TR1	752	214	761	408
TR1	736	540	602	395
TR1	635	401	125	549
TR1	127	458	172	215
TR1	39	257	874	559
TR1	859	406	220	323
TR1	295	90	0	1
TR1	307	70	1	1
TR2	607	349	816	404
TR2	568	573	304	556
TR2	138	320	108	155
TR2	265	78	718	61
TR2	810	137	814	222
TR2	463	266	406	326
TR2	392	384	332	151
TR2	391	215	229	321
TR2	578	309	2	1
TR2	777	436	1	1
TR3	468	48	310	246
TR3	225	343	352	441
TR3	97	666	129	854
TR3	207	698	411	894
TR3	560	794	641	604
TR3	717	416	629	350
TR3	682	183	43	373
TR3	738	612	37	459
TR3	571	225	2	1
TR3	578	206	1	1
TR4	529	386	447	261
TR4	345	133	162	122
TR4	124	257	192	454
TR4	83	579	41	741
TR4	173	865	224	779
TR4	309	714	455	733
TR4	18	306	458	892
TR4	247	667	197	425
TR4	423	415	2	1
TR4	431	392	1	1
TR5	753	301	575	297
TR5	413	336	170	404
TR5	248	264	418	95
TR5	680	204	658	88
TR5	1164	59	1049	202
TR5	986	268	1173	341
TR5	1150	412	48	87
TR5	623	104	893	231
TR5	820	365	0	1
TR5	750	357	2	1
TR6	648	456	663	588
TR6	594	714	496	612
TR6	371	500	250	636
TR6	89	588	92	413
TR6	159	240	291	141
TR6	422	227	600	135
TR6	694	208	417	483
TR6	277	642	186	311
TR6	682	370	0	1
TR6	697	388	1	1
TR7	180	570	77	440
TR7	120	220	133	64
TR7	435	277	587	449
TR7	792	215	727	137
TR7	879	204	960	386
TR7	663	633	569	666
TR7	544	558	412	390
TR7	761	428	417	61
TR7	244	640	0	1
TR7	220	639	2	1
TR8	704	374	818	496
TR8	805	621	994	404
TR8	751	140	455	255
TR8	289	148	59	129
TR8	268	396	135	458
TR8	470	605	567	564
TR8	940	447	523	262
TR8	777	222	350	489
TR8	790	560	0	1
TR8	811	554	2	1
TR9	643	88	750	190
TR9	528	437	698	469
TR9	752	553	534	595
TR9	404	623	99	608
TR9	184	467	131	268
TR9	117	134	388	185
TR9	429	262	447	530
TR9	427	351	263	390
TR9	652	13	0	1
TR9	697	363	1	1

[1Simo1]

Maybe it's only a coincidence for this particular track (TR7), but if I take these 40 bytes two for two and consider it as a coordinate for a point, I notice that most of them are in track, and first thirteen "points" are also sequentially along the lap! Also, other three points are on track, another two are slightly off track (but if you are in these point you can have part of car on track)

In attachment: yellow points are start points (these are sure and verified now 😄 ) , white points are the sequence of "points" on track and on correct path of lap, red points are on track but after previous sequence, green points are off track. There are files with points and with lines.

tr7_test.zip

[enriquesomolinos]

Hi,

Yes, you are probably right :). I revised the code and the first array I comment earlier to you contains 32 values, as you see, it appears to be points. The other array at this moment is unkown for me, but this values can perfectly be points.

Offset (byte)	Size (bytes)	Information
0	4	ima/mas file height or width
4	4	ima/mas file height or width
8	4	vai file zones
12	4*3	driver 1 start position w /driver 1 start position h /?
24	4*3	driver 2 start position w /driver 2 start position h /?
36	4*3	driver 3 start position w /driver 3 start position h /?
48	4*3	driver 4 start position w /driver 4 start position h /?
60	32*4	array points
188	80*4	array points

[1Simo1]

Let me understand 😄

If you have 32 * 4 and my assumption about two bytes points is correct, we have 64 points for first array? So maybe 160 points for second?

Or maybe we have 32 lines, each passing through two points?

[enriquesomolinos]

Hi,

The last array of points can be the position of the public!

Can you check this?

The decompiled code is very confusing in this aspect, but the game uses this point to draw the pedestrinans

[enriquesomolinos]

The first array stores 16 points, because each point have 2 coordinates and each coordinate uses 4 bytes.

Sorry for my english, I dont explain my ideas as well as in spanish.

The second array contains 40 points (40 pedestrians)

[1Simo1]

The decompiled code is very confusing in this aspect, but the game uses this point to draw the pedestrinans

Really, the decompiled code is confusing in almost all aspects 😁 and you're doing an excellent work!

Should I try 64 and 160 points or 32 and 80 respectively ?

[1Simo1]

About final part of file : I have checked first six values and I think you are right, these are the position of pedestrians! (First two near start line, other four along above very short straight). First two integers of four are a point, others maybe type of pedestrian (?)

This means that for each track there are 20 pedestrians.

Track File	value	value	value	value
TR7	244	640	0	1
TR7	220	639	2	1
TR7	184	128	0	1
TR7	205	132	0	1
TR7	322	127	1	1
TR7	345	135	2	1
TR7	588	358	2	1
TR7	623	364	0	1
TR7	646	368	1	1
TR7	816	198	1	1
TR7	838	210	0	1
TR7	820	219	2	1
TR7	818	282	1	0
TR7	843	301	0	1
TR7	720	540	1	1
TR7	702	544	1	1
TR7	383	538	2	1
TR7	360	540	0	1
TR7	345	640	1	1
TR7	318	634	2	1

[enriquesomolinos]

mmmm, yes you are right, there is only 20 pedestrians, the third and fourth number probably is the pedestrian image rotation.

So, the inf file only have 2 unkonw items, the pedestrian rotation and the starting grip point third value.