ZD2: bytes 0x8-0xF - Githubissues

Anyone has any ideas about the data represented by the bytes 0x8-0xF in the ZD2 files? These are the 8 (mostly non-zero) bytes following the file magic string.

These bytes appear to be different across different effects, though It's not very obvious what is encoded there. In fact these bytes are among the few which are different between different versions of a given ZD2. For example:

filename	version	bytes [0x8-0xF]
AIR.ZD2	v1.50	`[19 EB FF B4 07 00 00 00]`
AIR.ZD2	v1.40	`[89 4B 53 4B 03 00 00 00]`
AIR.ZD2	v1.30	`[D3 19 D6 C4 03 00 00 00]`

I'd love to believe that it's just some random non-initialized memory but given it's position and peculiarity between versions there is likely some specific meaning to these. Some ideas I considered so far without success:

checksum (md5, crc32); in general it's not clear what could be checksum'ed here, since the contained BMP icon, the ELF payload, and descriptions appear to remain the same between the versions in this case.
some bit-field, perhaps describing pedal models this ZD2 is supported on. Updates are often to add models but it's not obvious how those are encoded.
a bunch of some integer values (it does not appear to be floating-point bytes); well, that's not very meaningful

Love that you are asking these questions, I suspect that every Byte of the ZD2 has meaning...

'Air' is used pretty on every pedal.

$ find -name "list_sorted.txt" -exec grep 0x398a8d8aea4072acb0174117630e0f91 {} \;
0x09000010 : Air (v1.50, 6.40%), 0x398a8d8aea4072acb0174117630e0f91, ./A1 FOUR/unzipped/AIR.ZD2
0x09000010 : Air (v1.50, 6.40%), 0x398a8d8aea4072acb0174117630e0f91, ./A1X FOUR/unzipped/AIR.ZD2
0x09000010 : Air (v1.50, 6.40%), 0x398a8d8aea4072acb0174117630e0f91, ./G1X FOUR/unzipped/AIR.ZD2
0x09000010 : Air (v1.50, 6.40%), 0x398a8d8aea4072acb0174117630e0f91, ./G11/unzipped/AIR.ZD2
0x09000010 : Air (v1.50, 6.40%), 0x398a8d8aea4072acb0174117630e0f91, ./B1X FOUR/unzipped/AIR.ZD2
0x09000010 : Air (v1.50, 6.40%), 0x398a8d8aea4072acb0174117630e0f91, ./G1 FOUR/unzipped/AIR.ZD2
0x09000010 : Air (v1.50, 6.40%), 0x398a8d8aea4072acb0174117630e0f91, ./B1 FOUR/unzipped/AIR.ZD2
0x09000010 : Air (v1.50, 6.40%), 0x398a8d8aea4072acb0174117630e0f91, ./G3n/unzipped/AIR.ZD2
0x09000010 : Air (v1.50, 6.40%), 0x398a8d8aea4072acb0174117630e0f91, ./B3n/unzipped/AIR.ZD2
0x09000010 : Air (v1.50, 6.40%), 0x398a8d8aea4072acb0174117630e0f91, ./G6/unzipped/AIR.ZD2
0x09000010 : Air (v1.50, 6.40%), 0x398a8d8aea4072acb0174117630e0f91, ./G5n/unzipped/AIR.ZD2
0x09000010 : Air (v1.50, 6.40%), 0x398a8d8aea4072acb0174117630e0f91, ./G3Xn/unzipped/AIR.ZD2

So the Bytes that you are questioning are in the first line...

$ hexdump -C "./G1X FOUR/unzipped/AIR.ZD2" | head
00000000  5a 44 4c 46 78 00 00 00  19 eb ff b4 07 00 00 00  |ZDLFx...........|
00000010  01 00 01 00 01 00 00 00  80 00 00 00 00 00 00 00  |................|
00000020  00 00 00 00 00 00 00 00  80 00 00 00 00 00 00 00  |................|

And is the first 4 bytes of the 'Unknown' section.

$ python3 ../decode_effect.py -d "./G1X FOUR/unzipped/AIR.ZD2"       
Container: 
    length = 120
    hexdump = hexundump("""
    0000   19 EB FF B4 07 00 00 00 01 00 01 00 01 00 00 00   ................
    0010   80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
    0020   80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
    0030   80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
    0040   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
    0050   01                                                .
    """)

    unknown = b'\x19\xeb\xff\xb4\x07\x00\x00\x00\x01\x00\x01\x00\x01\x00\x00\x00'... (truncated, total 81)
    version = u'1.50' (total 4)
    group = 9
    id = 150994960
    aname = u'Air' (total 3)
    bname = b"Air\x00\x00\x00\x00\x00\xae'=" (total 11)
    name = u'Air' (total 3)
    groupname = u'REVERB' (total 6)
    hex3 = hexundump("""
    0000   00 5A 27 09 1B 98                                 .Z'...
    """)

    unknown3 = b"\x00Z'\t\x1b\x98" (total 6)
    unknown4 = Container: 
        unknown4 = ListContainer: 
            0
            1
            0
            1
            0
            0
            0
            0
    ICON = Container: 
        length = 182
        data = b'BM\xb6\x00\x00\x00\x00\x00\x00\x00>\x00\x00\x00(\x00'... (truncated, total 182)
    TXJ1 = Container: 
        length = 56
        data = b'\x95\x94\x89\xae\x96\xc2\x82\xe8\x82\xcc\x8b\xf3\x8bC\x8a\xb4'... (truncated, total 56)
    TXE1 = Container: 
        length = 75
        description = u'This effect reproduces the ambie'... (truncated, total 75)
    INFO = Container: 
        length = 20
        data = b'\x00\x00\x00\x00REVREVERB\x00\x00\x00' (total 16)
        dspload = 16.0
    DATA = Container: 
        length = 13169
        data = b'\x7fELF\x01\x01\x01@\x00\x00\x00\x00\x00\x00\x00\x00'... (truncated, total 13169)
    PRMJ = Container: 
        length = 796
        data = b'{  \r\n\t"Parameter'... (truncated, total 796)
    PRME = Container: 
        length = 715
        xml = u'{  \r\n\t"Parameters":[  \r\n\t\t{  \r\n\t'... (truncated, total 715)

The reason I mention all of this, is that it becomes trivial to decode ALL of the effects and then compare the various bits of them.

There is no duplication in the effects of the G1 Four, and eye-balling it the values are evenly spread so this could very likely be some for of checksum (if so it's probably CRC32 in the same way that the midi link uses).

$ find G1\ FOUR/unzipped/ -name '*.ZD2' -exec python3  ../decode_effect.py -d {} \; | grep -A 3 "length" | grep '0000 ' | sort | uniq -c
      1     0000   02 FA 90 8C 02 00 00 00 01 00 00 00 00 00 00 00   ................
      1     0000   03 D6 EA 31 07 00 00 00 01 00 00 00 00 00 00 00   ...1............
      1     0000   04 DB 8F 8F 07 00 00 00 01 00 00 00 00 00 00 00   ................
      1     0000   06 47 DA 68 07 00 00 00 01 00 01 00 00 00 00 00   .G.h............
      1     0000   07 44 34 BA 02 00 00 00 01 00 01 00 01 00 00 00   .D4.............
      1     0000   08 29 C0 19 07 00 00 00 01 00 01 00 00 00 00 00   .)..............
      1     0000   0D 6F 1F 31 07 00 00 00 01 00 01 00 01 00 00 00   .o.1............
      1     0000   12 47 DD 99 07 00 00 00 01 00 00 00 00 00 00 00   .G..............
      1     0000   18 3D 15 81 07 00 00 00 01 00 00 00 00 00 00 00   .=..............
      1     0000   19 EB FF B4 07 00 00 00 01 00 01 00 01 00 00 00   ................
      1     0000   1A 78 06 2A 02 00 00 00 01 00 00 00 00 00 00 00   .x.*............
      1     0000   1C C6 96 5E 02 00 00 00 01 00 01 00 00 00 00 00   ...^............
      1     0000   1D AC 20 08 02 00 00 00 01 00 01 00 00 00 00 00   .. .............
      1     0000   1E A0 16 F5 07 00 00 00 01 00 01 00 00 00 00 00   ................
      1     0000   1E A1 BC 59 07 00 00 00 01 00 00 00 00 00 00 00   ...Y............
      1     0000   1E DF 86 B4 07 00 00 00 01 00 00 00 00 00 00 00   ................
      1     0000   1E EE 8E 05 02 00 00 00 01 00 00 00 00 00 00 00   ................
      1     0000   22 6F F6 A2 07 00 00 00 01 00 01 00 01 00 00 00   "o..............
      1     0000   22 81 10 45 07 00 00 00 01 00 01 00 01 00 00 00   "..E............
      1     0000   28 57 E5 3D 06 00 00 00 01 00 01 00 00 00 00 00   (W.=............
      1     0000   29 C9 B6 85 07 00 00 00 01 00 01 00 01 00 00 00   )...............
      1     0000   2C 29 9D 43 07 00 00 00 01 00 00 00 00 00 00 00   ,).C............
      1     0000   2C 55 BB 05 07 00 00 00 01 00 01 00 00 00 00 00   ,U..............
      1     0000   2D 42 7F D3 07 00 00 00 01 00 01 00 00 00 00 00   -B.............
      1     0000   2D DF 99 BB 07 00 00 00 01 00 01 00 01 00 00 00   -...............
      1     0000   33 55 4E FF 07 00 00 00 01 00 01 00 00 00 00 00   3UN.............
      1     0000   36 1E BD 6D 07 00 00 00 01 00 01 00 01 00 00 00   6..m............
      1     0000   39 AF 48 30 02 00 00 00 01 00 01 00 00 00 00 00   9.H0............
      1     0000   3B 3C 05 3D 07 00 00 00 01 00 01 00 00 00 00 00   ;<.=............
      1     0000   3C 8C 56 58 07 00 00 00 01 00 00 00 00 00 00 00   <.VX............
      1     0000   3D 53 A8 B3 07 00 00 00 01 00 01 00 00 00 00 00   =S..............
      1     0000   3D F4 B6 F5 07 00 00 00 01 00 01 00 00 00 00 00   =...............
      1     0000   3E 7D 7E E4 07 00 00 00 01 00 01 00 01 00 00 00   >}~.............
      1     0000   43 76 92 81 07 00 00 00 01 00 00 00 00 00 00 00   Cv..............
      1     0000   44 7A 7F 83 07 00 00 00 01 00 01 00 01 00 00 00   Dz.............
      1     0000   44 B5 E1 61 07 00 00 00 01 00 01 00 01 00 00 00   D..a............
      1     0000   49 CB 71 D1 07 00 00 00 01 00 01 00 01 00 00 00   I.q.............
      1     0000   4A 51 8A 3C 02 00 00 00 01 00 00 00 00 00 00 00   JQ.<............
      1     0000   4B 58 2D E7 07 00 00 00 01 00 01 00 00 00 00 00   KX-.............
      1     0000   50 D1 FE 33 07 00 00 00 01 00 01 00 00 00 00 00   P..3............
      1     0000   53 DD 07 30 07 00 00 00 01 00 01 00 01 00 00 00   S..0............
      1     0000   55 ED 8C 50 02 00 00 00 01 00 00 00 00 00 00 00   U..P............
      1     0000   56 65 DE C1 07 00 00 00 01 00 01 00 01 00 00 00   Ve..............
      1     0000   58 CB D4 A3 02 00 00 00 01 00 00 00 00 00 00 00   X...............
      1     0000   59 16 9A FD 02 00 00 00 01 00 00 00 00 00 00 00   Y...............
      1     0000   59 AE 0D 19 02 00 00 00 01 00 01 00 00 00 00 00   Y...............
      1     0000   59 D2 48 0C 07 00 00 00 01 00 00 00 00 00 00 00   Y.H.............
      1     0000   59 DB 3A 1A 02 00 00 00 01 00 01 00 01 00 00 00   Y.:.............
      1     0000   5B D1 75 30 06 00 00 00 01 00 01 00 00 00 00 00   [.u0............
      1     0000   5C B3 09 36 07 00 00 00 01 00 01 00 01 00 00 00   \..6............
      1     0000   5E 7F 04 EC 02 00 00 00 01 00 01 00 01 00 00 00   ^..............
      1     0000   5E B1 BD E3 06 00 00 00 01 00 01 00 00 00 00 00   ^...............
      1     0000   60 14 22 2B 02 00 00 00 01 00 01 00 01 00 00 00   `."+............
      1     0000   61 3A 95 04 07 00 00 00 01 00 01 00 00 00 00 00   a:..............
      1     0000   61 E4 56 DD 07 00 00 00 01 00 00 00 01 00 00 00   a.V.............
      1     0000   64 A8 EB 84 07 00 00 00 01 00 00 00 00 00 00 00   d...............
      1     0000   68 30 2E D3 07 00 00 00 01 00 00 00 00 00 00 00   h0..............
      1     0000   6C F1 56 23 07 00 00 00 01 00 01 00 00 00 00 00   l.V#............
      1     0000   73 75 98 13 07 00 00 00 01 00 01 00 01 00 00 00   su..............
      1     0000   74 79 D2 27 07 00 00 00 01 00 01 00 00 00 00 00   ty.'............
      1     0000   74 F2 C4 40 06 00 00 00 01 00 01 00 00 00 00 00   t..@............
      1     0000   79 42 A6 A8 02 00 00 00 01 00 00 00 00 00 00 00   yB..............
      1     0000   7A 00 70 73 07 00 00 00 01 00 00 00 00 00 00 00   z.ps............
      1     0000   80 7D C7 1A 02 00 00 00 01 00 00 00 00 00 00 00   .}..............
      1     0000   82 D1 C2 F1 07 00 00 00 01 00 00 00 00 00 00 00   ................
      1     0000   82 D9 35 C7 07 00 00 00 01 00 01 00 00 00 00 00   ..5.............
      1     0000   87 F4 D8 B2 07 00 00 00 01 00 01 00 00 00 00 00   ................
      1     0000   88 10 4B DA 07 00 00 00 01 00 01 00 00 00 00 00   ..K.............
      1     0000   88 86 E7 48 07 00 00 00 01 00 01 00 01 00 00 00   ...H............
      1     0000   8A 04 63 AC 07 00 00 00 01 00 01 00 00 00 00 00   ..c.............
      1     0000   93 77 BC 89 07 00 00 00 01 00 01 00 00 00 00 00   .w..............
      1     0000   97 15 22 A3 07 00 00 00 01 00 01 00 00 00 00 00   ..".............
      1     0000   97 27 78 E6 07 00 00 00 01 00 00 00 00 00 00 00   .'x.............
      1     0000   9A D0 49 76 07 00 00 00 01 00 01 00 01 00 00 00   ..Iv............
      1     0000   9B 51 01 5D 07 00 00 00 01 00 01 00 01 00 00 00   .Q.]............
      1     0000   9C 40 F7 EB 07 00 00 00 01 00 01 00 00 00 00 00   .@..............
      1     0000   9D 3E 64 C0 02 00 00 00 01 00 00 00 00 00 00 00   .>d.............
      1     0000   9E E1 60 A0 07 00 00 00 01 00 00 00 00 00 00 00   ..`.............
      1     0000   A0 46 8A 4D 06 00 00 00 01 00 01 00 00 00 00 00   .F.M............
      1     0000   A1 AD FD DE 07 00 00 00 01 00 01 00 01 00 00 00   ................
      1     0000   A2 76 BE 5C 07 00 00 00 01 00 01 00 00 00 00 00   .v.\............
      1     0000   A3 92 39 ED 07 00 00 00 01 00 00 00 00 00 00 00   ..9.............
      1     0000   A8 CF 22 71 07 00 00 00 01 00 01 00 00 00 00 00   .."q............
      1     0000   AC 63 90 7E 07 00 00 00 01 00 01 00 00 00 00 00   .c.~............
      1     0000   AD 31 82 D9 07 00 00 00 01 00 00 00 00 00 00 00   .1..............
      1     0000   AD E6 AF EF 07 00 00 00 01 00 00 00 00 00 00 00   ................
      1     0000   AE 97 88 44 07 00 00 00 01 00 01 00 00 00 00 00   ...D............
      1     0000   B2 05 FC 60 02 00 00 00 01 00 01 00 00 00 00 00   ...`............
      1     0000   B3 EF C4 50 07 00 00 00 01 00 01 00 00 00 00 00   ...P............
      1     0000   B5 BB E2 99 02 00 00 00 01 00 00 00 00 00 00 00   ................
      1     0000   B6 35 D8 5D 02 00 00 00 01 00 00 00 00 00 00 00   .5.]............
      1     0000   B8 A0 ED 32 07 00 00 00 01 00 01 00 01 00 00 00   ...2............
      1     0000   B9 E1 F5 34 07 00 00 00 01 00 00 00 00 00 00 00   ...4............
      1     0000   BB 56 30 E9 07 00 00 00 01 00 01 00 00 00 00 00   .V0.............
      1     0000   BE B4 92 B7 07 00 00 00 01 00 00 00 01 00 00 00   ................
      1     0000   C0 AE 22 11 02 00 00 00 01 00 00 00 00 00 00 00   ..".............
      1     0000   C1 25 A6 72 07 00 00 00 01 00 00 00 00 00 00 00   .%.r............
      1     0000   C6 68 92 99 06 00 00 00 01 00 01 00 01 00 00 00   .h..............
      1     0000   C8 24 8A DE 02 00 00 00 01 00 01 00 01 00 00 00   .$..............
      1     0000   C8 B7 80 65 07 00 00 00 01 00 01 00 00 00 00 00   ...e............
      1     0000   C9 CC 02 71 07 00 00 00 01 00 01 00 01 00 00 00   ...q............
      1     0000   CB 3D 75 3D 02 00 00 00 01 00 01 00 00 00 00 00   .=u=............
      1     0000   CE DB 2E DB 07 00 00 00 01 00 00 00 00 00 00 00   ................
      1     0000   CF 2F E2 57 07 00 00 00 01 00 00 00 00 00 00 00   ./.W............
      1     0000   CF 49 EF 99 07 00 00 00 01 00 00 00 00 00 00 00   .I..............
      1     0000   D0 19 4F 8F 07 00 00 00 01 00 01 00 00 00 00 00   ..O.............
      1     0000   D1 1F 46 39 07 00 00 00 01 00 00 00 00 00 00 00   ..F9............
      1     0000   D4 12 69 ED 07 00 00 00 01 00 01 00 00 00 00 00   ..i.............
      1     0000   D5 64 6A 51 06 00 00 00 01 00 01 00 00 00 00 00   .djQ............
      1     0000   D7 7C 43 2A 07 00 00 00 01 00 01 00 01 00 00 00   .|C*............
      1     0000   DA 1D AC 76 07 00 00 00 01 00 00 00 00 00 00 00   ...v............
      1     0000   DC 89 DC D8 07 00 00 00 01 00 01 00 00 00 00 00   ................
      1     0000   DD D2 E1 5A 07 00 00 00 01 00 00 00 00 00 00 00   ...Z............
      1     0000   DE FF F2 7E 02 00 00 00 01 00 00 00 00 00 00 00   ...~............
      1     0000   E3 04 E6 48 07 00 00 00 01 00 00 00 00 00 00 00   ...H............
      1     0000   E4 AD 9D 5C 07 00 00 00 01 00 01 00 00 00 00 00   ...\............
      1     0000   E5 A7 E0 B9 07 00 00 00 01 00 00 00 01 00 00 00   ................
      1     0000   E6 0D A2 91 02 00 00 00 01 00 01 00 00 00 00 00   ................
      1     0000   E6 51 3B 7B 07 00 00 00 01 00 00 00 00 00 00 00   .Q;{............
      1     0000   E7 8D 4B 80 07 00 00 00 01 00 00 00 00 00 00 00   ..K.............
      1     0000   EC 50 AB 3D 07 00 00 00 01 00 01 00 00 00 00 00   .P.=............
      1     0000   EF 75 65 84 02 00 00 00 01 00 01 00 00 00 00 00   .ue.............
      1     0000   F3 EB 0F 72 07 00 00 00 01 00 01 00 01 00 00 00   ...r............
      1     0000   F4 02 F8 22 07 00 00 00 01 00 01 00 00 00 00 00   ..."............
      1     0000   F6 08 D4 02 07 00 00 00 01 00 01 00 00 00 00 00   ................
      1     0000   F6 2D B0 B3 06 00 00 00 01 00 01 00 00 00 00 00   .-..............
      1     0000   F8 18 0A C9 07 00 00 00 01 00 01 00 00 00 00 00   ................
      1     0000   F9 7D 83 4D 07 00 00 00 01 00 01 00 01 00 00 00   .}.M............
      1     0000   FB 96 86 E9 07 00 00 00 01 00 00 00 00 00 00 00   ................
      1     0000   FC E2 14 F3 07 00 00 00 01 00 01 00 00 00 00 00   ................

Also note that the values you mentioned are followed by a byte, for the different versions of Air this is either 03 or 07 which is with an extra bit set... would be interesting to figure out what that means too.

I was looking for something that denotes whether an effect is mono/stereo for the input/output.

0x02: 0b0000 0010
0x03: 0b0000 0011
0x07: 0b0000 0111

It does seem like a bit-field or perhaps a part of the bigger bit-field. Looks like in your sorted sample 0x07 is more prevalent.

'Air' is used pretty on every pedal.

Actually, it's listed for most but not all -- it's not listed directly for B6, H8, R20, A1 D, A1X D; the latter two have no effects at all listed in the 7-list.

If those 4 bytes are a CRC32, then it's possible to brute force by calculating CRC for different sections/lengths and comparing. I previously did this on another project.

crc_find.py.txt crc_find2.py.txt

Note: there are different CRC32 methods, but 'crcmod' can handle them all.

Confirmed, these are a CRC32 checksum. Of the Bytes 12 through to -16 bytes from end of file. I'll add this functionality to the main code. crc_check.py.txt

v1.5$ python3 crc_check.py
00000000: 5A 44 4C 46 78 00 00 00  19 EB FF B4 07 00 00 00  ZDLFx...........
None
Stated checksum 0xb4ffeb19
inverted 0x4b0014e6
Inverted Match @ 4B0014E6

v1.4$ python3 ../v1.5/crc_check.py 
00000000: 5A 44 4C 46 78 00 00 00  89 4B 53 4B 03 00 00 00  ZDLFx....KSK....
None
Stated checksum 0x4b534b89
inverted 0xb4acb476
Inverted Match @ B4ACB476

Done, can we close this bug?

v1.4$ python3 ../../decode_effect.py -V -o temp.zd2 -v AIR.ZD2
Checksum Validated: 0x4b534b89
Checksum Recalculated: 0x4b534b89

v1.5$ python3 ../../decode_effect.py -V -o temp.zd2 -v AIR.ZD2
Checksum Validated: 0xb4ffeb19
Checksum Recalculated: 0xb4ffeb19

Seems not all the effects pass the checksum validation....

$ grep -A 1 "Invalid" checksum_validation.txt 
Checksum Invalid: 0x3c4ebb35
0x01000080 : OptComp (v1.50, 12.72%), ./A1 FOUR/unzipped/OPTCOMP.ZD2
--
Checksum Invalid: 0x3c4ebb35
0x01000080 : OptComp (v1.50, 12.72%), ./A1X FOUR/unzipped/OPTCOMP.ZD2
--
Checksum Invalid: 0xa0d735f6
0x0b000030 : ChromeWah (v1.50, 6.97%), ./A1X FOUR/unzipped/CHRM_WAH.ZD2
--
Checksum Invalid: 0x3c4ebb35
0x01000080 : OptComp (v1.50, 12.72%), ./G1X FOUR/unzipped/OPTCOMP.ZD2
--
Checksum Invalid: 0xa0d735f6
0x0b000030 : ChromeWah (v1.50, 6.97%), ./G1X FOUR/unzipped/CHRM_WAH.ZD2
--
Checksum Invalid: 0x1f67920a
0x05000018 : MS4x12GB (v1.30, 9.02%), ./G1X FOUR/unzipped/MS4X12GB.ZD2
--
Checksum Invalid: 0x2384e91e
0x0500002b : FD MA2x12 (v1.30, 9.10%), ./G1X FOUR/unzipped/FDMA2X12.ZD2
--
Checksum Invalid: 0x8b69af12
0x05000040 : MK1 1x12 (v1.30, 8.84%), ./G1X FOUR/unzipped/MK1_1X12.ZD2
--
Checksum Invalid: 0xc8a2c0bc
0x030000f0 : BG GRID (v1.30, 16.40%), ./G1X FOUR/unzipped/BG_GRID.ZD2
--
Checksum Invalid: 0x3c4ebb35
0x01000080 : OptComp (v1.50, 12.72%), ./G11/unzipped/OPTCOMP.ZD2
--
Checksum Invalid: 0xa0d735f6
0x0b000030 : ChromeWah (v1.50, 6.97%), ./G11/unzipped/CHRM_WAH.ZD2
--
Checksum Invalid: 0x1f67920a
0x05000018 : MS4x12GB (v1.30, 9.02%), ./G11/unzipped/MS4X12GB.ZD2
--
Checksum Invalid: 0x2384e91e
0x0500002b : FD MA2x12 (v1.30, 9.10%), ./G11/unzipped/FDMA2X12.ZD2
--
Checksum Invalid: 0x8b69af12
0x05000040 : MK1 1x12 (v1.30, 8.84%), ./G11/unzipped/MK1_1X12.ZD2
--
Checksum Invalid: 0xb4385f36
0x06000170 : Duo Phase (v1.10, 9.84%), ./G11/unzipped/DUO_PHA.ZD2
--
Checksum Invalid: 0xc8a2c0bc
0x030000f0 : BG GRID (v1.30, 16.40%), ./G11/unzipped/BG_GRID.ZD2
--
Checksum Invalid: 0x3c4ebb35
0x01000080 : OptComp (v1.50, 12.72%), ./B1X FOUR/unzipped/OPTCOMP.ZD2
--
Checksum Invalid: 0x3c4ebb35
0x01000080 : OptComp (v1.50, 12.72%), ./G1 FOUR/unzipped/OPTCOMP.ZD2
--
Checksum Invalid: 0x1f67920a
0x05000018 : MS4x12GB (v1.30, 9.02%), ./G1 FOUR/unzipped/MS4X12GB.ZD2
--
Checksum Invalid: 0x2384e91e
0x0500002b : FD MA2x12 (v1.30, 9.10%), ./G1 FOUR/unzipped/FDMA2X12.ZD2
--
Checksum Invalid: 0x8b69af12
0x05000040 : MK1 1x12 (v1.30, 8.84%), ./G1 FOUR/unzipped/MK1_1X12.ZD2
--
Checksum Invalid: 0xc8a2c0bc
0x030000f0 : BG GRID (v1.30, 16.40%), ./G1 FOUR/unzipped/BG_GRID.ZD2
--
Checksum Invalid: 0x3c4ebb35
0x01000080 : OptComp (v1.50, 12.72%), ./B1 FOUR/unzipped/OPTCOMP.ZD2
--
Checksum Invalid: 0x3c4ebb35
0x01000080 : OptComp (v1.50, 12.72%), ./G3n/unzipped/OPTCOMP.ZD2
--
Checksum Invalid: 0xa0d735f6
0x0b000030 : ChromeWah (v1.50, 6.97%), ./G3n/unzipped/CHRM_WAH.ZD2
--
Checksum Invalid: 0x1f67920a
0x05000018 : MS4x12GB (v1.30, 9.02%), ./G3n/unzipped/MS4X12GB.ZD2
--
Checksum Invalid: 0x2384e91e
0x0500002b : FD MA2x12 (v1.30, 9.10%), ./G3n/unzipped/FDMA2X12.ZD2
--
Checksum Invalid: 0x8b69af12
0x05000040 : MK1 1x12 (v1.30, 8.84%), ./G3n/unzipped/MK1_1X12.ZD2
--
Checksum Invalid: 0xb4385f36
0x06000170 : Duo Phase (v1.10, 9.84%), ./G3n/unzipped/DUO_PHA.ZD2
--
Checksum Invalid: 0xc8a2c0bc
0x030000f0 : BG GRID (v1.30, 16.40%), ./G3n/unzipped/BG_GRID.ZD2
--
Checksum Invalid: 0x3c4ebb35
0x01000080 : OptComp (v1.50, 12.72%), ./R20/unzipped/OPTCOMP.ZD2
--
Checksum Invalid: 0x1f67920a
0x05000018 : MS4x12GB (v1.30, 9.02%), ./R20/unzipped/MS4X12GB.ZD2
--
Checksum Invalid: 0x2384e91e
0x0500002b : FD MA2x12 (v1.30, 9.10%), ./R20/unzipped/FDMA2X12.ZD2
--
Checksum Invalid: 0x8b69af12
0x05000040 : MK1 1x12 (v1.30, 8.84%), ./R20/unzipped/MK1_1X12.ZD2
--
Checksum Invalid: 0xc8a2c0bc
0x030000f0 : BG GRID (v1.30, 16.40%), ./R20/unzipped/BG_GRID.ZD2
--
Checksum Invalid: 0x3c4ebb35
0x01000080 : OptComp (v1.50, 12.72%), ./B3n/unzipped/OPTCOMP.ZD2
--
Checksum Invalid: 0x3c4ebb35
0x01000080 : OptComp (v1.50, 12.72%), ./G6/unzipped/OPTCOMP.ZD2
--
Checksum Invalid: 0xa0d735f6
0x0b000030 : ChromeWah (v1.50, 6.97%), ./G6/unzipped/CHRM_WAH.ZD2
--
Checksum Invalid: 0x1f67920a
0x05000018 : MS4x12GB (v1.30, 9.02%), ./G6/unzipped/MS4X12GB.ZD2
--
Checksum Invalid: 0x2384e91e
0x0500002b : FD MA2x12 (v1.30, 9.10%), ./G6/unzipped/FDMA2X12.ZD2
--
Checksum Invalid: 0x8b69af12
0x05000040 : MK1 1x12 (v1.30, 8.84%), ./G6/unzipped/MK1_1X12.ZD2
--
Checksum Invalid: 0xc8a2c0bc
0x030000f0 : BG GRID (v1.30, 16.40%), ./G6/unzipped/BG_GRID.ZD2
--
Checksum Invalid: should be 0x3c4ebb35
0x01000080 : OptComp (v1.50, 12.72%), ./G5n/unzipped/OPTCOMP.ZD2
--
Checksum Invalid: should be 0xa0d735f6
0x0b000030 : ChromeWah (v1.50, 6.97%), ./G5n/unzipped/CHRM_WAH.ZD2
--
Checksum Invalid: should be 0x1f67920a
0x05000018 : MS4x12GB (v1.30, 9.02%), ./G5n/unzipped/MS4X12GB.ZD2
--
Checksum Invalid: should be 0x2384e91e
0x0500002b : FD MA2x12 (v1.30, 9.10%), ./G5n/unzipped/FDMA2X12.ZD2
--
Checksum Invalid: should be 0x8b69af12
0x05000040 : MK1 1x12 (v1.30, 8.84%), ./G5n/unzipped/MK1_1X12.ZD2
--
Checksum Invalid: should be 0xb4385f36
0x06000170 : Duo Phase (v1.10, 9.84%), ./G5n/unzipped/DUO_PHA.ZD2
--
Checksum Invalid: should be 0xc8a2c0bc
0x030000f0 : BG GRID (v1.30, 16.40%), ./G5n/unzipped/BG_GRID.ZD2
--
Checksum Invalid: should be 0x3c4ebb35
0x01000080 : OptComp (v1.50, 12.72%), ./H8/unzipped/OPTCOMP.ZD2
--
Checksum Invalid: should be 0x1f67920a
0x05000018 : MS4x12GB (v1.30, 9.02%), ./H8/unzipped/MS4X12GB.ZD2
--
Checksum Invalid: should be 0x2384e91e
0x0500002b : FD MA2x12 (v1.30, 9.10%), ./H8/unzipped/FDMA2X12.ZD2
--
Checksum Invalid: should be 0x8b69af12
0x05000040 : MK1 1x12 (v1.30, 8.84%), ./H8/unzipped/MK1_1X12.ZD2
--
Checksum Invalid: should be 0xc8a2c0bc
0x030000f0 : BG GRID (v1.30, 16.40%), ./H8/unzipped/BG_GRID.ZD2
--
Checksum Invalid: should be 0x3c4ebb35
0x01000080 : OptComp (v1.50, 12.72%), ./G3Xn/unzipped/OPTCOMP.ZD2
--
Checksum Invalid: should be 0xa0d735f6
0x0b000030 : ChromeWah (v1.50, 6.97%), ./G3Xn/unzipped/CHRM_WAH.ZD2
--
Checksum Invalid: should be 0x1f67920a
0x05000018 : MS4x12GB (v1.30, 9.02%), ./G3Xn/unzipped/MS4X12GB.ZD2
--
Checksum Invalid: should be 0x2384e91e
0x0500002b : FD MA2x12 (v1.30, 9.10%), ./G3Xn/unzipped/FDMA2X12.ZD2
--
Checksum Invalid: should be 0x8b69af12
0x05000040 : MK1 1x12 (v1.30, 8.84%), ./G3Xn/unzipped/MK1_1X12.ZD2
--
Checksum Invalid: should be 0xb4385f36
0x06000170 : Duo Phase (v1.10, 9.84%), ./G3Xn/unzipped/DUO_PHA.ZD2
--
Checksum Invalid: should be 0xc8a2c0bc
0x030000f0 : BG GRID (v1.30, 16.40%), ./G3Xn/unzipped/BG_GRID.ZD2

...Confirmed, these are a CRC32 checksum. Of the Bytes 12 through to -16 bytes from end of file. I'll add this functionality to the main code.

Nice find. Can you give an explicit example of the matching bytes and the CRC32 result? Say, you can use any of the AIR.ZD2 versions.

In general this indeed could be the case, as the trailing 16 bytes of ZD2 are the ones which are also differing between the versions of the effect's ZD2. However, while these are different between v1.30 and v1.40, these trailing bytes are the same between v1.40 and v1.50.

filename	version	trailing bytes [0x3B5D-0x3B6C]
AIR.ZD2	v1.50	`[29 23 BE 84 E1 6C D6 AE 52 90 49 F1 F1 BB E9 EB]`
AIR.ZD2	v1.40	`[29 23 BE 84 E1 6C D6 AE 52 90 49 F1 F1 BB E9 EB]`
AIR.ZD2	v1.30	`[5F C5 17 C5 3E FC 33 63 C3 84 92 AB 08 A3 AA 3F]`

I tried to make sense out of the CRC32 matching you've done just to understand the actual values in ZD2 that match.

To recap:

ZD2 file is split into 3 parts:

head:[0x00:0x0b], the last 4-bytes store a crc32_checksum value
body:[0x0c:(end-0x10)]
tail:[(end-0x0F):end], the last 16-bytes of the ZD2 file

CRC32 checksum is calculated over the body
resulting CRC32 checksum value is inverted: crcValue ^ 0xFFFFFFFF
the inverted CRC32 value is matching the 4-byte integer at the end of header section: crc32_checksum

tail -c +13 -${version}/${zd2_file} | head -c -16 > zd2_body.bin
crc32 zd2_body.bin

For "AIR.ZD2" effect this yields:

filename	version	bytes [0x8:0xB]	body [0x0C:(end-0x10)]	CRC32(body)	CRC32 ^`0xFFFFFFFF`
AIR.ZD2	v1.50	`[19 EB FF B4]`	`[07 ... 0A]`	`0x4B0014E6`	`0xB4FFEB19`
AIR.ZD2	v1.40	`[89 4B 53 4B]`	`[03 ... 0A]`	`0xB4ACB476`	`0x4B534B89`
AIR.ZD2	v1.30	`[D3 19 D6 C4]`	`[03 ... 0A]`	`0x3B29E62C`	`0xC4D619D3`

... Seems not all the effects pass the checksum validation

I looked into OPTCOMP.ZD2 versions.

Curiously, two recent versions (1.40, 1.50) list the same values in the checksum field ([f9 7d 83 4d]), yet these have different inverted-CRC32 checksum for their "body" (0x490f713c, 0x3c4ebb35). Neither matches. While the checksums in the file version 1.30 seem to match according to the algorithm above.

The only two parts where the v1.40 and v1.50 differ are the version string and the first byte of the "body" ( 0x03 vs. 0x07) which, as we established, seems to be some kind of bit-field.

Similar situation is with MS4X12GB.ZD2 (v1.20 vs. 1.30)

colordiff -y <(xxd v1.20/MS4X12GB.ZD2) <(xxd v1.30/MS4X12GB.ZD2)

00000000: 5a44 4c46 7800 0000 cf2f e257 0300 0000  ZDLFx..../ | 00000000: 5a44 4c46 7800 0000 cf2f e257 0700 0000  ZDLFx..../
00000010: 0100 0000 0000 0000 8000 0000 0000 0000  ..........   00000010: 0100 0000 0000 0000 8000 0000 0000 0000  ..........
...
00000050: 0000 0000 0000 0000 0131 2e32 3000 0005  .........1 | 00000050: 0000 0000 0000 0000 0131 2e33 3000 0005  .........1

Not sure if this is just a human edit error or the algorithm above takes some additional details. As an idea, you may try to check if all of the mismatching effects/versions as well have 0x07 value following the checksum bytes.

Your understanding of the CR32 coding is correct.

Further the body is split in to the sections which start with the markers "ICON", "TXJ1", "TXE1", "INFO", "DATA", "PRMJ" and "PRME".

I pushed a couple of quick changes to help experimentation, though since I was already creating ZD2 by patching others together I don't think that the pedal actually checks the CRC32. Since we know about it, it's better to correct the parameter when writing ZD2 file....

I also suspect that the incorrect CRC32 seen in the official ZD2 is some sloppiness within Zoom. Maybe they had to fix something, and took a shortcut?

While most of the "Invalid" have 07, a couple have 05. And there are lots of "Valid" which have 07. I don't think these things are connected. checksum_validation.txt

Well, 0x05 and 0x07 both have the upper bit set, unlike 0x03. It's quite possible that the upper bit describes the type of the update. In those cases that have 0x07 (that I checked) the update introduced by the version was just the version number bump, that is there was no actual code/description changes. Meanwhile, versions with 0x03 (or lower??) do have differences in the code sections.

In that sense, the checksum may be directly pointing to the version of the effect's code, almost like a version-control hash. Though it's not clear why would one want to promote the version number without actual code changes.

When I mentioned that maybe Zoom took a shortcut, I meant that maybe an engineer just fixed an issue with the data by hand editing the file (or maybe skipped some part of the process to re-create the checksum).... these things happen in production.

I'm not convinced that changing the parts of the file that the CRC32 applies would have any benefit. I can run the same "find the checksum-ed block" test on a Invalid checksum, to see if it's another block of the file... I doubt it.

I'm under the impression that sections of the ZD2 (other than 'code'), are actually just for GuitarLab to do it's thing (display ICON, description, etc). I don't know whether the pedal actually does anything with them.... can check by changing and seeing if pedal behavior remains the same.

mungewell / zoom-zt2

ZD2: bytes 0x8-0xF #48