Reversing 'zptc' patch files

[mungewell]

As of yesterday I implemented the ability to download/upload the patch files via the command line tool.

zoom-zt2-master-20200513>python zoomzt2.py -p 10 test.zptc
zoom-zt2-master-20200513>python -m hexdump test.zptc | head
00000000: 50 54 43 46 34 01 00 00  01 00 00 00 05 00 00 00  PTCF4...........
00000010: 10 00 00 00 00 00 00 00  00 00 4D 53 20 48 69 47  ..........MS HiG
00000020: 61 69 6E 20 40 00 00 03  11 00 00 04 10 00 00 05  ain @...........
00000030: 40 00 00 01 70 00 00 09  54 58 4A 31 2C 00 00 00  @...p...TXJ1,...
00000040: 4D 53 20 38 30 30 82 F0  8E 67 82 C1 82 BD 83 58  MS 800...g.....X
00000050: 83 5E 83 93 83 5F 81 5B  83 68 82 C8 98 63 82 DD  .^..._.[.h...c..
00000060: 83 54 83 45 83 93 83 68  81 42 00 00 54 58 45 31  .T.E...h.B..TXE1
00000070: 2C 00 00 00 54 68 69 73  20 73 74 61 6E 64 61 72  ,...This standar
00000080: 64 20 64 69 73 74 6F 72  74 69 6F 6E 20 73 6F 75  d distortion sou
00000090: 6E 64 20 75 73 65 73 20  4D 53 20 38 30 30 2E 00  nd uses MS 800..

I am looking to figure out/document this file format, if anyone has a link to some info that would be very helpful.

[mungewell]

The 'zptc' patch contains markers for blocks - PTCF TXJ1, TXE1, EDTB, PPRM

The 'PTCF' block contains the Int32ul ID's for the selected effects. TXE1 contains a textual description.

The 'EDTB' block appears to contain the parameters/settings for each of the 5 effects as 5 block of 24 bytes. These bytes are NOT simple/clear, they appear to be unaligned encoded bit stream (similar to that used in USB HID, I guess).

Attempting to read settings from device with single effect in patch.

FX: Defret
p1 = sense, 0..30
p2 = colour, 1..10
p3 = tone, 1..50
p4 = vol, 0..100

Turning 1st dial
        0000   81 00 00 CF 00 24 40 07 9C 00 00 00 00 00 00 00   .....$@.........
        0000   81 00 00 0F 01 24 40 07 9C 00 00 00 00 00 00 00   .....$@.........
                        ^  ^^
Turning 2nd dial
        0000   81 00 00 8F 07 04 40 07 9C 00 00 00 00 00 00 00   ......@.........
        0000   81 00 00 8F 07 0C 40 07 9C 00 00 00 00 00 00 00   ......@.........
                              ^^
Turning 3rd dial
        0000   81 00 00 8F 07 0C 40 07 9C 00 00 00 00 00 00 00   ......@.........
        0000   81 00 00 8F 07 0C 00 0A 9C 00 00 00 00 00 00 00   ................
                                 ^^ ^^ why both?
Turning 4th dial (30, 4, 41, 100)
        0000   81 00 00 8F 07 0C 00 0A 00 00 00 00 00 00 00 00   ................
        0000   81 00 00 8F 07 0C 00 0A 90 01 00 00 00 00 00 00   ................
                                       ^^ ^^

Looks like an encoded bitstream (similar to USB HID stream), remember low nyble is sent first!

0, 10, 0, 100
        0000   81 00 00 0F 00 24 00 00 90 01 00 00 00 00 00 00   .....$..........
                        ^^ ^^ ^^ ^^ ^^ ^^ ^^
Bitstream LowBitFirst:  1111,0000 0000,0000 0010,0100 0000,0000 0000,0000 0000,1001 1000,0000
                                                                            ^^ ^^^^ ^^ 0x64 = 100 
                                              ^^ ^^^^ ^^ 0x09 = 9th value, ie 10

30, 10, 50, 100
        0000   81 00 00 8F 07 24 40 0C 90 01 00 00 00 00 00 00   .....$@.........
                        ^^ ^^ ^^ ^^ ^^ ^^ ^^
                        1111,0001 1110,0000 0010,0100 0000,0010 0011,0000 0000,1001 1000,0000
                                                                            ^^ ^^^^ ^^ 0x64 = 100 
                                                             ^^ ^^^^ ^^ 0x31 = 49th value, ie 50
                                              ^^ ^^^^ ^^ 0x09th value, ie 10
                               ^^ ^^^^ ^^ 0x1e = 30

30, 1, 50, 0
        0000   81 00 00 8F 07 00 40 0C 00 00 00 00 00 00 00 00   ......@.........
                        ^^ ^^ ^^ ^^ ^^ ^^ ^^
                        1111,0001 1110,0000 0000,0000 0000,0010 0011,0000 0000,0000 0000,0000
                                                                            ^^ ^^^^ ^^ 0x00 = 0 
                                                             ^^ ^^^^ ^^ 0x31 = 49th value, ie 50
                                              ^^ ^^^^ ^^ 0x00 = 0th value, ie 1
                               ^^ ^^^^ ^^ 0x1e = 30

15, 5, 26, 50
        0000   81 00 00 CF 03 10 40 06 C8 00 00 00 00 00 00 00   ......@.........
                        ^^ ^^ ^^ ^^ ^^ ^^ ^^
                        1111,0011 1100,0000 0000,1000 0000,0010 0110,0000 0001,0011 0000,0000
                                                                            ^^ ^^^^ ^^ 0x32 = 50
                                                             ^^ ^^^^ ^^ 0x1a = 26
                                              ^^ ^^^^ ^^ 0x04 = 4th value, ie 5
                               ^^ ^^^^ ^^ 0x0f = 15

Note sure whether the earlier bytes define the format or whether the 'ZD2' effect file needs to be parsed to find it...

[mungewell]

I'm wonder whether there is any encoding.... of the samples I've looked at it appears to be 12bit per parameter for the first 5 parameters, and then 8bits per parameter.

For example:

FX: Z-Syn
p1 = Freq, 0..10
p2 = Range, 0..20
p3 = Decay, 0..100
p4 = Reso, 0..20
p5 = Wave, Saw/Sqr
p6 = Tone, 0..10
p7 = Bal, 0..100
p8 = Vol, 0..100

5, 11, 58, 11, Sqr, 5, 56, 100
        0000   63 00 00 4F 01 2C 80 0E 2C 40 00 14 E0 90 01 00   c..O.,..,@......
                        ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^
1111,0010 1000,0000 0011,0100 0000,0001 0111,0000 0011,0100 0000,0010 0000,0000 0010,1000 0000,0111 0000,1001 1000,0000
                 -- --          -- --          -- --          -- --          -- --
                                                                                                      ^^ ^^^^ ^^ 0x64 = 100
                                                                                            ^^ ^^^^ ^^ 0x38 = 56
                                                                                  ^^ ^^^^ ^^ 0x05
                                                                   ^^ ^^^^ ^^ 0x01
                                                    ^^ ^^^^ ^^ 0x0b = 11
                                     ^^ ^^^^ ^^ 0x3a = 58
                      ^^ ^^^^ ^^ 0x0b = 11
       ^^ ^^^^ ^^ 0x05

[mungewell]

The bytes at the start of each of these 24-byte blocks was assumed to be config

FX: TapeEcho (4 parameters)
        0000   61 00 00 D0 8B B8 00 0E 00 00 00 00 00 00 00 00   a...............
               ^^ ^^ ^^  ^ config?
               1000,0110 0000,0000 0000,0000 0000

The '1st' bit is enabled/disabled, and it looks like the others are a shifted form of the effect ID (as used in FLST_SEQ)

     TapeEcho  1000,0110 0000,0000 0000,0000 0000 -i  0x8000030  
     LoopRoll  1100,0110 0000,0000 0000,0000 0111 -i  0x7000041
     Z-Syn     1100,0110 0000,0000 0000,0000 1111 -i  0x7800031
     GtGEQ7    1001,0101 0000,0000 0000,0000 0010 -i  0x2000054
     GtGEQ     1100,0101 0000,0000 0000,0000 0010 -i  0x2000051 
     A-PanDly  1100,0101 1000,0000 0000,0000 0000 -i  0x80000d1
     MS800     1100,0100 0000,0000 0000,0000 0001 -i  0x4000011
     DualDly   1100,0010 1000,0000 0000,0000 0000 -i  0x80000a1
     PhaseDly  1100,0011 1000,0000 0000,0000 0000 -i  0x80000e1 
     DuoPhase  1100,0111 0100,0000 0000,0000 0011 -i  0x6000171

Which means that to figure out the number of effects, size of each parameter, etc... you need to look else where.

[mungewell]

I was confused by several 'zptc' files having multiple 'PPRM' records in them, I think that this is corruption leading back to original FW image having overlapping data.

For example this section of '133' file... which is clearly wrong:

00008fa0  00 00 00 00 00 00 00 00  50 50 52 4d 0c 00 00 00  |........PPRM....|
00008fb0  00 00 00 00 00 00 00 00  32 00 00 00 73 6f 75 6e  |........2...soun|
00008fc0  64 2c 20 77 68 69 63 68  20 69 73 20 73 75 69 74  |d, which is suit|
00008fd0  61 62 6c 65 20 66 6f 72  20 64 72 6f 70 20 74 75  |able for drop tu|
00008fe0  6e 69 6e 67 73 20 61 6e  64 20 66 6f 72 20 72 65  |nings and for re|
00008ff0  63 6f 72 64 69 6e 67 2e  45 44 54 42 78 00 00 00  |cording.EDTBx...|
00009000  81 00 00 02 00 90 01 19  00 00 00 00 00 00 00 00  |................|

Once these patch locations are written by guitar lab they are scrubbed clean, and only have 1 record of each type

[mungewell]

So a lot more is now known, I've just be lazy in documenting.... and uploading code.

simon@thevoid:~/zoom-zt2-sdw-github$ python3 zoomzt2.py -p 10 -R patch_10.zptc
simon@thevoid:~/zoom-zt2-sdw-github$ hexdump -C patch_10.zptc
00000000  50 54 43 46 08 01 00 00  01 00 00 00 05 00 00 00  |PTCF............|
00000010  10 00 00 00 00 00 00 00  00 00 4d 53 20 48 69 47  |..........MS HiG|
00000020  61 69 6e 20 40 00 00 03  11 00 00 04 10 00 00 05  |ain @...........|
00000030  40 00 00 01 70 00 00 09  54 58 4a 31 00 00 00 00  |@...p...TXJ1....|
00000040  54 58 45 31 2c 00 00 00  54 68 69 73 20 73 74 61  |TXE1,...This sta|
00000050  6e 64 61 72 64 20 64 69  73 74 6f 72 74 69 6f 6e  |ndard distortion|
00000060  20 73 6f 75 6e 64 20 75  73 65 73 20 4d 53 20 38  | sound uses MS 8|
00000070  30 30 2e 00 45 44 54 42  78 00 00 00 81 00 00 c6  |00..EDTBx.......|
00000080  03 c8 40 0d f0 00 00 00  00 00 00 00 00 00 00 00  |..@.............|
00000090  00 00 00 00 23 00 00 48  00 d0 00 0c 18 01 0f 2c  |....#..H.......,|
000000a0  19 01 00 00 00 00 00 00  00 00 00 00 21 00 00 0a  |............!...|
000000b0  00 90 81 07 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000000c0  00 00 00 00 81 00 00 42  00 70 80 04 00 00 00 00  |.......B.p......|
000000d0  00 00 00 00 00 00 00 00  00 00 00 00 e1 00 00 52  |...............R|
000000e0  0c 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000000f0  00 00 00 00 50 50 52 4d  0c 00 00 00 00 00 00 00  |....PPRM........|
00000100  00 00 00 00 11 84 0c 00  00 00 00 00 00 00 00 00  |................|
00000110  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000002f8

simon@thevoid:~/zoom-zt2-sdw-github$ python3 decode_preset.py -d patch_10.zptc > patch_10.zptc.txt

patch_10.zptc.txt

[shooking]

I'm wonder whether there is any encoding.... of the samples I've looked at it appears to be 12bit per parameter for the first 5 parameters, and then 8bits per parameter.

For example:

FX: Z-Syn
p1 = Freq, 0..10
p2 = Range, 0..20
p3 = Decay, 0..100
p4 = Reso, 0..20
p5 = Wave, Saw/Sqr
p6 = Tone, 0..10
p7 = Bal, 0..100
p8 = Vol, 0..100

5, 11, 58, 11, Sqr, 5, 56, 100
        0000   63 00 00 4F 01 2C 80 0E 2C 40 00 14 E0 90 01 00   c..O.,..,@......
                        ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^
1111,0010 1000,0000 0011,0100 0000,0001 0111,0000 0011,0100 0000,0010 0000,0000 0010,1000 0000,0111 0000,1001 1000,0000
                 -- --          -- --          -- --          -- --          -- --
                                                                                                      ^^ ^^^^ ^^ 0x64 = 100
                                                                                            ^^ ^^^^ ^^ 0x38 = 56
                                                                                  ^^ ^^^^ ^^ 0x05
                                                                   ^^ ^^^^ ^^ 0x01
                                                    ^^ ^^^^ ^^ 0x0b = 11
                                     ^^ ^^^^ ^^ 0x3a = 58
                      ^^ ^^^^ ^^ 0x0b = 11
       ^^ ^^^^ ^^ 0x05

This is pretty similar to the format for B1On. First 4 bytes relate to FXID (bit 0 of 1st byte determines if FX is on). Byte 2 《 8 + byte 1 seems to be some function of the FX. On the B1On the 3rd and 4th bytes seems very strange mapping to the Group ID. So many time nybble of byte 4 is 2x the group ID. But sometimes it seems (byte3 & 0xF) + (byte 4 & 0xF) / 2 is the GID.

Then as you found the next few parameters are 12 bit packed. Then a gap then 12 bit again.

5 x FX followed by volume and some currently unknown stuff then the patch name.

The only way I have found to get at FX params is by parsing the firmware update file. I am close bjr probably need to read your Python for clues. I haven't worked out how to read the FLST on B1On yet.

It seems as FX are modified on the pedal the whole FX is updated.

[hooham482]

Does anybody happen to know where in .zptc file is a device ID? I mean, G3n files are not allowed in B3n and vice versa. There should be something that Guitar Lab reads and then makes "alien" files unable to load.

[shooking]

So the FLST_SEQ.ZDT has the ZD2 filenames, their group and FXID, the version.

MORE DETAILS on Zoom B1XFour

I am rediscovering some of what Mungewell has done - for example in this thread I see some additional options that were not in the readme. So it looks like Mungewell already inferred the max midi values - I independently worked this out recently having never consciously noted that was being done.

What I have done is to write my own C++ to derive info, but this weekend I hacked at Mungewell's zoomzt2.py. I use it stock to get the ZT2 list of ZD2, version, IDS

Then I added an option (maybe it already exists) to get individual ZD2s. I derive the FX name ID GROUPID GroupName Description Version Parameters: Name, description, default value, max value (midi) Which I dump in JSON format Bitmap which I dump as filename.zs2.bmp

[shooking]

Something I noticed (but I only have B1On, B1XFour G1XFour) - isn't the number after the zptc the version? So the above are 4. Some other seem to talk about version 6. I would hope the official Zoom code would look at such versions for compatibility.

[sslag]

Can I use this script to decrypt the parameters of the zptc patch so I can create it by hand on the G1four? Thanks!

[shooking]

Yes. Plenty of ways to approach it. So you can use @mungewell code to generate the values, then load the FX onto machine, set values on pedal. Or you could use my code on ZoomPedalFun (it is on git) to set values.

Once you set it save the values on your pedal. I could probably write some JSON to allow values to be imported. Not sure how much you program etc - the b1xfour.py program is based on Mungewell's zoom-zt2. The difference is the patch format needs to be adjusted based on pedal (Each pedal tells us the patch size, bank size etc).

I then parse the encoded files into JSON. The FX and the patches.

Right now I zero it out on reboot but we could just as easy allow import from JSON. If the FX didn't exist we.can flag.l it. There are various videos on my YouTube (also referenced on the readme and wiki of ZoomPedalFun thar demonstrate aspects of what you want).

I am quite maxed out at the moment but I do plan to write a parser to strip out certain G5n FX and try to keep.the essence of a patch for playing on a G1XFour or B1XFour (recall G5n can have up to 9 FX whereas G1XFour seems to have 7 max but more limited processing power).

[mungewell]

@sslag As @shooking mentioned you can --dump or --summary the contents of a zptc patch.

$ python3 zoomzt2.py -p 11 patch_11.zptc

$ ls -al patch_11.zptc 
-rw-rw-r-- 1 simon simon 760 Jan 25 13:28 patch_11.zptc

$ python3 decode_preset.py -s patch_11.zptc
Name: ZEP 1959  
Effect 1: 0x0B000020
   Enabled: False
   Param 1: 50
   Param 2: 50
   Param 3: 0
   Param 4: 80
   Param 5: 0
   Param 6: 0
   Param 7: 0
   Param 8: 0
Effect 2: 0x04000019
   Enabled: True
   Param 1: 7
   Param 2: 69
   Param 3: 67
   Param 4: 40
   Param 5: 64
   Param 6: 30
   Param 7: 60
   Param 8: 2
Effect 3: 0x01000040
   Enabled: True
   Param 1: 0
   Param 2: 100
   Param 3: 30
   Param 4: 0
   Param 5: 0
   Param 6: 0
   Param 7: 0
   Param 8: 0
Effect 4: 0x05000018
   Enabled: True
   Param 1: 0
   Param 2: 50
   Param 3: 49
   Param 4: 26
   Param 5: 0
   Param 6: 0
   Param 7: 0
   Param 8: 0
Effect 5: 0x08000030
   Enabled: True
   Param 1: 2006
   Param 2: 21
   Param 3: 40
   Param 4: 0
   Param 5: 0
   Param 6: 0
   Param 7: 0
   Param 8: 0

The Effect # is a combined Group and ID that Zoom uses to track which effect, and the parameters are sequentially listed. The G1Four has 3 parameters per screen, larger pedals often split the parameters across multiple screen.

May I ask why you want to transcribe the file by hand, you should be able to uploaded them directly?

[mungewell]

I realized that there isn't a easy way to know the Group/ID numbers, so I wrote a script to parse them out of the known ZD2 effects.... summary list is here: https://github.com/mungewell/zoom-zt2/blob/master/zoom_fx/master.txt

$ python3 decode_preset.py -s patch_11.zptc | grep Effect
Effect 1: 0x0B000020
Effect 2: 0x04000019
Effect 3: 0x01000040
Effect 4: 0x05000018
Effect 5: 0x08000030

$ grep 0x08000030 zoom_fx/master.txt 
0x08000030 : TapeEcho (v1.40)
$ grep 0x05000018 zoom_fx/master.txt 
0x05000018 : MS4x12GB (v1.20)
$ grep 0x01000040 zoom_fx/master.txt 
0x01000040 : ZNR (v1.40)
$ grep 0x04000019 zoom_fx/master.txt 
0x04000019 : MS 1959 (v1.00)
$ grep 0x0b000020 zoom_fx/master.txt 
0x0b000020 : BlackWah (v1.40)

[sslag]

First of all thanks for your answers. I really only need to decode some patch to load it by hand, I like to learn how the sound changes as I add effects and set their options from their default values. That's fine for me.

Now I have to apologize, because I think my problem is that I'm new to python:

zoom-zt2-master$ python3 decode_preset.py -d Tool.zptc
Traceback (most recent call last):
  File "decode_preset.py", line 7, in <module>
    from construct import *
ModuleNotFoundError: No module named 'construct'

zoom-zt2-master# pip install constructs
Collecting constructs
No matching distribution found for constructs

I use Ubuntu 18.04 python 3.6.7 Thanks

[shooking]

Check out my videos on YouTube. I show how to do this and in description I mention how to pip install - especially on a Raspberry Pi.

And with my GUI you can modify values in real time and hear them. Of course tonelib and Zoom Guitar Lab can do this but not of you use @Mungewell's code to add new FX.

You might also like the recent video I did on why some FX cannot load on G1XFour from G5n - I show the guard zt2 that Mungewell's discovered

[sslag]

shooking please link to your videos

[shooking]

https://youtu.be/3JMdiGSC3ZI

That should get you to the series. Maybe look at the GCE-3 emulating a B1XFour or G5n rendered with my software.

I am in UK timezone - if u want direct help I am sure we can find a way to communicate.

[mungewell]

@sslag The error you showed is due to a missing dependency, you will need to install Construct module. The pre-built exe's have this included, but other scripts need it installed separately.

#--------------------------------------------------
# Define ZT2/ZD2 file format using Construct (v2.9)
# requires:
# https://github.com/construct/construct

This module lets me describe how the bits/bytes of the various binary blobs are used. The definition is bi-directional, meaning that you can both decode and encode the objects. For example mid way in the decode_patch script the binary is converted to a python dictionary (see with --dump), but by using --output you can rebuilt the binary object from this dictionary.

This means that you can change the dictionary and then rebuild patch - for example if you wanted to specify different Group/ID numbers for effects. Maybe this is useful if you want to map all G3N (2 screen) effects to the equivalent G1Four (1 screen) effects - do their parameters align??

[shooking]

hi @mungewell

When you say _I realized that there isn't a easy way to know the Group/ID numbers, so I wrote a script to parse them out of the known ZD2 effects.... summary list is here: https://github.com/mungewell/zoom-zt2/blob/master/zoom_fx/master.txt_

You know the ID is encoded in the ZD2 itself right? In your master list you have some where the difference is an FX is on v off - which makes me think you pulled this out of the patches? Just checking you know it is 0x40 in a ZDL and 0x60 offset in a ZD2 ie 30 00 00 01 for Slow Attack (FXID 0x30, GID 0x8, group 1).

[mungewell]

The master.txt is auto generated from the ZD2 files referenced in the allZDL6.lst file. I downloaded the them all and extracted information. The script files show how this is done. The last column is the MD5 of the ZD2 file(s).

The difference in the ID value is because of the 1 display variants, I made some code to automatically switch patches between 1 and 2 screen effects here: https://github.com/mungewell/zoom-zt2/blob/master/decode_preset.py#L96

[mungewell]

Closing as 'zptc' are mostly known now, and we've moved on.