Strange patch data

[Colatino]

Hi! First of all, thanks for all the work you and @mungewell have done to decrypt the Zoom sysex commands.

I have a G1X Four (f/w v2.0) and have been trying to get data from the EDTB block as I'm thinking on building a Raspberry Pi Pico USB pedal controller with 5 switches to turn individual effects on and off.

I've read a lot of yours and @mungewell discussions and findings and have been successful in decoding patch info from .zptc files. But when it comes to decode data from the pedal itself (using the getcurrentpatch sysex) I'm getting some weird data, sometimes the data comes with multiple "PPRM" blocks, sometimes the block's names have null bytes in the middle (like E.DTB or ED.TB or any other variation) and, worst of all, the EDTB block size sometimes doesn't seem to respect the 24*n_effects + 4 bytes. It is like some trash data is being inserted in the stream (Tonelib Zom works perfectly with the same hardware).

The problem seems to be consistent on the patch level, as no matter what USB cable and port I use, I always get the same data, identical with intruder null bytes and all.

My python script uses mido library (under windows 10):

import mido,time

zin=mido.open_input('ZOOM G Series 0')
zout=mido.open_output('ZOOM G Series 1')

in_editor=mido.Message('sysex',data=[0x52,0x00,0x6e,0x50])
get_cur_patch=mido.Message('sysex',data=[0x52,0x00,0x6e,0x29])
out_editor=mido.Message('sysex',data=[0x52,0x00,0x6e,0x51])
flush=mido.Message('sysex',data=[0x52,0x00,0x6e,0x60,0x05,0x00])

bypass=mido.Message('sysex',data=[0x52,0x00,0x6e,0x64,0x0b])
effect1_on=mido.Message('sysex',data=[0x52 ,0x00 ,0x6E ,0x64 ,0x03 ,0x00 ,0x00 ,0x00 ,0x00 ,0x02 ,0x00 ,0x00 ,0x00])
effect1_off=mido.Message('sysex',data=[0x52 ,0x00 ,0x6E ,0x64 ,0x03 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00])

def getCurPatch():
    global m
    zout.send(in_editor)
    _=zin.receive()
    zout.send(get_cur_patch)
    m=zin.receive()    
    zout.send(out_editor)
    _=zin.receive()    
    parsePatch(m.bin())

def parsePatch(data):
    global effects,t0    
    edtb=data.split(b'EDTB')[1].split(b'PPRM')[0]    
    effects={'id':[],'st':[]}
    eff_st=[]
    neff=int(edtb[0]/24)
    t0=time.perf_counter_ns()
    for i in range(neff):    
        bits=''        
        uniao=edtb[7+i*24]<<24 | edtb[6+i*24]<<16 | edtb[5+i*24]<<8 | edtb[4+i*24]        
        if not ((uniao>>1) & 0xfffffff == 1):
            effects['id'].append((uniao>>1) & 0xfffffff)
            effects['st'].append(uniao & 1)    

getCurPatch()

t1=time.perf_counter_ns()   

print((t1-t0)/1000,'us')
print(effects)

I've attached a file with the raw data I got from the pedal hoping that you could give me some some light on that matter.

tsorg.zip

[shooking]

It doesn't look like you are unpacking the sysex? Sysex is 8 bits for start and stop - 7 for rest. 8 bytes get used to represent 7 8 bit bytes. It seems to me you read in the current patch but forget to transform it. Check out Mungewell's zoom-zt2 especially the unpack method or my C++

[shooking]

Ha ... I see I didn't check it in ...soz. I will try to check it in later this weekend. But since you are in Python likely you will reuse Mungewell's unpack method?

[mungewell]

The 'fractured' text seems like you might be receiving the data in the packed form, where 8bit data is packed into 7bits over midi. I have functions for handling that here: https://github.com/mungewell/zoom-zt2/blob/master/zoomzt2.py#L232

When pulling patches out of the FW image, I saw that sometimes the patch had extra crap after it; I deduced that a new (smaller) patch was written over the data of an old (larger) patch... leaving the crap there. Perhaps you are getting the same from the pedal.

Also, you can get information about the display, and the effects in various slots. Check decode_screens.py.

Enabling individual effects in the patch and their parameters can be done via sysex: https://github.com/mungewell/zoom-zt2#effects-settings

[Colatino]

I wasn't really expecting to get answers so fast!

It doesn't look like you are unpacking the sysex? Sysex is 8 bits for start and stop - 7 for rest. 8 bytes get used to represent 7 8 bit bytes. It seems to me you read in the current patch but forget to transform it. Check out Mungewell's zoom-zt2 especially the unpack method or my C++

I didn't know about the unpacking part (I have never tinkered with MIDI before), that must be why I got the data right from the zptc files but with "trash" from MIDI.

@mungewell The multiple 'PPRM' parts being from an old larger patch makes complete sense.

[Colatino]

Just confirmed that unpacking solved the strange data issue. Just copy pasted @mungewell unpacking method.

Do you know if there is any SysEx to cycle screens (Memory -> Stomp -> Edit)?

Right now I'm using python to test and prototype but I intend to migrate to c (arduino style) because the RPi Pico doesn't have normal Python, instead it has micropython (or circuitpython) and those have little support for MIDI and zero support for the construct library.

I like the idea of getting info of the current patch so that I'll be able to toggle some leds to see the enabled status of the fxs.

I'll check the decode_screens.py , thanks for the recommendation.

[shooking]

I agree with your approach - I got into Python late in life and am more comfortable with C but when I see the elegance of @mungewell's code I have to agree it is a great prototyping tool - especially in his hands.

What I did was to create a GUI for use on Pi4 with a 22" touch screen. No need to solder stuff etc, handles Midi and you can run a ton of other stuff as well if you like.

This "chip shortage" means even such Pi4 are overpriced. Not sure how powerful a Pico is - or how powerful it needs to be to achieve what you want

This "Memory -> Screens -> Edit" is not possible AFAIK from Midi commands. Maybe if you soldered onto the Zoom itself? But why would you want to do this? Check out my https://github.com/shooking/ZoomPedalFun/blob/main/python/b1xfour001.py This fires off a fork of Mungewell's zoomzt-2 to gather the current FX/Patch status of a pedal (works on B/G/1/(X)/Four, G3n, G5n - not tests on B3n but I gotta figure it will work - and GCE-3 in any mode).

Then it GUI's the patches - including any added FX - something I have been trying to get ToneLib to look into with us. You can then manipulate the values in real time (touch screen is a good option here). Crude but saves me bending over too much.

Sometimes you have to try to start it 3 times - I will work on that part. The pedals is sometimes busy and wont respond.

I will add the C/C++ to decode the patch format. I have used STL on Arduino Mega in the past - so I figure such code should work - but again a Pi4 with touchscreen => no need to adding knobs/dials etc - do it in software. With a 1080p 22" real estate you can make some useful GUIs?

[shooking]

Oh and if you are open about your code drop me a link and I will add it to the README/Wiki - you have seen the wiki with the pedal specific sysex write ups? If you find any new ones or better explanations drop me a line and I will update it. If you think of other features - same deal. I need to make some time to look at the B1On/MS CDR70 and G5. I made a lot of progress but not added it yet. As @mungewell suggests - if you want to read the raw firmware you have to account for 4096 block size as described by @barsik. On the older pedals it is the only way - @mungewell discovered newer pedals can dump their firmware via midi - I independently stumbled on similar stuff, Googles and found the source :-)

[Colatino]

What I did was to create a GUI for use on Pi4 with a 22" touch screen. No need to solder stuff etc, handles Midi and you can run a ton of other stuff as well if you like.

I'm trying to achieve what something like they did in these videos ( https://youtu.be/G-wxfuwe6EI?t=323 https://youtu.be/t4fpknwvxvs?t=168 https://m.facebook.com/toonystomp/videos/zoom-g1-four-footswitch-mod/713335869435539/?_se_imp=08bp2J4vIReLwlpQa ) but without soldering anything to the pedal itself while still being able to control with my foot, so touchscreen is not an option here. I'm not really concerned about bending too much as I want to use it only while playing and not when changing parameters.

Toggling (is this a word?) individual effects on/off is possible with the commands you already discovered and I know how to do that with the pico using its GPIOs.

This "Memory -> Screens -> Edit" is not possible AFAIK from Midi commands. Maybe if you soldered onto the Zoom itself? But why would you want to do this?

Being able to cycle screens with SysEx would be THE killer command for my purpose.

Oh and if you are open about your code drop me a link and I will add it to the README/Wiki - you have seen the wiki with the pedal ...

No problem sharing my code but right now it is only in my pc but I'll upload to github as soon as I have anything working with the RPi Pico as my python code is only a chopped up version of yours and @mungewell.

[shooking]

What would you want to do via screen cycling with your foot?

[Colatino]

What would you want to do via screen cycling with your foot?

Maybe Killer command was an overstatement, but it would surely give it a very useful funcionality as being able to use the pedal's own lcd to see the effects chain would allow me to see wich one I need to toggle. Edit: Without bending to cycle screens

Do you know what this one SysEx do (I know the pedal ID is not the G1 one)?

CTRL SW/PDL
f0 52 00 5a 31 06 06 00 00 f7

I couldn't find it in your docs but it is "documented" here (old repo) https://github.com/PedalController/MidiPedalController/blob/master/zoom/br/com/srmourasilva/multistomp/zoom/gseries/Specific.txt

[mungewell]

I wasn't really expecting to get answers so fast!

@Colatino For me (and I expect for @shooking) this is a 'passion project', and it's great to be able help others. I think your idea is great, and I'd encourage you to make a GitHub project! Even if it's just chopped up code (your words) it's good to put it together in one place. Add some instructions and pictures and you might even make it on to Hack-a-day. :-)

The packing isn't really a Midi thing, each manufacturer does their own thing under the banner of SysEx.

I'm not sure whether I've seen the ability to cycle screens, try enabling Editor mode on the device and then monitor the midi output. Does the pedal output anything when the button is pressed... if so these messages may give a clue on how to trigger.

We did figure out how to change some stuff, perhaps which screen is selected is done in a similar way. https://github.com/mungewell/zoom-zt2#effects-settings

[shooking]

That is for older pedal. 31 is parameter. I think 06 is like FX7? I documented the generic command on the wiki.

Regarding the FX list. You get that from the patch. Moving the order - again I documented the cycling command in its own wiki entry. The complication is you need to know that some FXs take up more than a slot. Hence why in my modification of Mungewell's zoom-zt2 I emit JSON with the slot width and parameters. So you could read patch. Derived FX ID order. Get name of each FX and know full parameters and width. Make your model of the pedal in your Pico. Display what you need. I found the BMPs and emit them. Mungewell added a neater more Pythonesque way to do it - so I merged his code back in and use that.

Not sure how you will display the bitmaps - hence why I went to touchscreen.

[mungewell]

The Bitmaps and XML are sections of the effect binary (ZD2) and can be extracted/cached. The effects have a 32bit ID which can be used to tie everything together, this is tells the patch which effects are loaded in what order - for example. decode_effect.py can help here.

The 'better' pedals can use 'wide' effects where they display across 2 screens, the G/B/A1's use the right most dial to page through... there is no reason your project couldn't show all the parameters at once.

On construct it's a tool I often overuse, sometimes it's a little clumsy though... and can be a pain. Like trying to get the data out of ETDB. The bit order was a pain, hence the auto-reverse stuff. Not elegant I'm afraid.

[shooking]

100% agree. I love reverse engineering stuff - who knew I am Asperger's. Most of what you suggest can / is done in the GUI I produced. But I am no artist. And am a tad colour blind. To be honest I got me a G5n and transferred all the Bass FX I need plus some acoustic models from A1X. Playing tomorrow morning and the G5n pretty much does everything I need. BUT the B1XFour (or if you saw video a G1XFour turned into B1XFour) has handy ability to run off battery or USB power brick.

We both look forward to what you do with your Zoom - and i figure the community would too. Again if u make any YT let me know and I will add it to my play list.

[Colatino]

The G5n is somewhat expensive for the average brazilian, the G1xFour is 1/5 of the price while still delivering great value.

I'm already very satisfied with what I learnt from you guys and will be able to build a somewhat cheap pedal-controller-pedal with 5 pushbuttons each toggling an individual fx of the currently selected patch.

One of my goals is to give the owners of the G1's a safer method of mimicking some of the G3 or G5 functionalities without having to actually do soldering and modifying the circuit board and risk ruining a piece of equipment that cost, for some that use it, about half a month's salary.

The other goal is to have fun with playing guitar, electronics and programing.

[shooking]

I also linked to Snr Mouza on my main wiki. He is also from Brazil and created Arduino based automation. You might want to connect with him? The big issue with all of these pedals is literally the pedal - we haven't found how to get info from it - without a pedal I guess you can only add a normal pedal between guitar and unit. The G3n and G3n can accept expression pedal. But no info leaked. The G5 has also ext pedal and you can modify what the pedal FX affects ... I not worked out if this carries across to the modern pedals.

I will try to write up what I found there as neither of us has 100% decoded every last bit in the sysex.

[Colatino]

Thanks both @mungewell and @shooking for all the help! I consider this specific issue solved but I still have some other questions I'll be asking on other threads.

[mungewell]

@Colatino You mentioned making the 'external button box' for enabling the effects of the patch.

Woke up this morning wondering whether you had noticed the Midi command to set the tempo? Not really wanting to give you more work to do on your project, but having a 'tap-tempo' button would be cool. ;-)

[Colatino]

Hey @shooking !

Yeah, I did notice that command but, apart from the looper and rhythm (drums), wich effect use this setting?

Implementing that for delays would require changing that fx "time" parameter, in fact it seems to be pretty easy to do. I think something like hold fx button for 2-sec to activate tap-tempo, tap the tempo, hold for 2-sec to confirm. Calculating the time between taps and sending the new parameter value.

I've given up on the idea of cycling screens and decided to build a version with some cheap $1 oled screens (0.91"), each one dedicated to a slot/button, this way I can see wich slot has wich effect.

[shooking]

Oh - do let me know which OLED you propose. I know another collaborator suggested LiquidInk which would also be low power.

I have been a bit maxed out with real work recently. I think I will open a discussion and welcome input on modelling the pedal on Python. More when I add the description.

I don't use tap tempo - I just set it to a value. And Inc/Decr with 10 second for 1 and 1 for slow might work?

[Colatino]

Yeah, I also don't use it, just set to a value and forget about it.

The screen I'm thinking:

Power usage is around 20mA with 50% of pixels lit

[mungewell]

Some of the effects can be set with BPM, rather than a ms time... denoted by little 'note' icon. https://zoomcorp.com/media/documents/E_G1FOUR_FX-list.pdf

Does that display have a protocol to change/assign different I2C address, ie can you have more than one display on the same I2C bus?

[Colatino]

Some of the effects can be set with BPM, rather than a ms time... denoted by little 'note' icon. https://zoomcorp.com/media/documents/E_G1FOUR_FX-list.pdf

Does that display have a protocol to change/assign different I2C address, ie can you have more than one display on the same I2C bus?

Usually these screens come with one of two addresses (3C or 3D), to use more than 2 one would need to use an I2C multiplexer, like the TCA9548A.