Open BuhJuhWuh opened 1 year ago
I am not sure about the reason for this. I guess: The message queue is somehow blocked. I have not tried continuous parameter changes in the Teensy (pedal) version of the communication to the THRII. But i have programmed a PC based version ( a program like Yamaha's THR-Remote, but in C# in Visual Studio and for my own purposes only). In this program i have sliders for all parameters including "Master". They work without any problems. So there could be a bug in the C++ code, that has not been noticed by me yet, because i have not sent continuous parameter changes there yet.
Thanks.
I'm not too familiar with the queue system. Does it wait to receive an Ack for a given message before allowing the next item in the queue to be sent? Or can it cope with overlapping messages (e.g. send 1st msg, send 2nd msg, receive 1st Ack, receive 2nd Ack)?
I'm still figuring out how to debug it, but it looks to me like it gets stuck in a loop waiting for an Ack that never comes back from the amp. I can't tell if that is because a) the pedal failed to send the msg out correctly, b) the amp didn't receive the msg, c) it did but failed to send an Ack, or d) the amp sent an Ack but the teensy didn't notice receiving it.
I wonder if this could be worked around by adding a timeout to break out of the loop...?
I expect your d) to be true.you could do a timeout for awaiting acks. This should be a duration, that exceeds any expected reaction time for a command.
I haven't tested it for very long yet, but that seems to have done the trick, thanks. The longest operations seemed be taking just short of 100ms in my system, so I built in a bit of headroom and it's lasted a lot longer than before without freezing up.
Next question: is it possible to change the Guitar Volume and Audio Volume parameters with any of your existing methods? You mention it in your sysex protocol doc, but I'm struggling to see how to achieve that.
I have updated the code with a timeout. Perhaps you did it yourself yet. But please have a look at my solution in "THR30II_Pedal.cpp" and try it with your pedal. I am interested, if the timeout gets active.
Thanks - that looks very similar to my solution. I will try yours out when I get a chance and report back (might be a few days).
I have updated the code with a timeout. Perhaps you did it yourself yet. But please have a look at my solution in "THR30II_Pedal.cpp" and try it with your pedal. I am interested, if the timeout gets active.
Hi Martin, From a quick check, not quite working yet. I think:
In case it is helpful, you can see my edited version of your code here, with:
elapsedMillis acktimer;
uint8_t acktimeout = 200;
defined earlier on.
As I say, I think this is working okay, but I haven't tested it extensively yet.
On a related question, is it possible to change the Guitar Volume and Audio Volume parameters with any of your existing methods? You mention it in your sysex protocol doc, but I'm struggling to see how to achieve that.
Cheers, Jo
Of course i had defined _time_stamp in THR30II.h, just missed to update the file in GitHub. The elapsedMillis type by Paul Stoffregen was new for me - I always do it manually with millis() and am used to this. So my code has to be a bit different than yours. But i admit, that "elapsedMillis" is worth a change in habits...
I had in mind to memorize the millis()-value when the message is sent out. If millis() has proceeded more than the timeout value (e.g. 250) the message is dequeued. I do not see a reason to reset the timer - (or do i think wrong here?)
The GuitarVolume and AudioVolume are parameters with the pseudo UnitKey 0xFFFF. In the protocol description it is shown how to read theses parameters out. Setting a value for them is just a parameter setting like Gain or Master but with the UnitKey 0xFFFF an the parameter keys like shown in the documentation (or better the key from the Symbol table for the actual firmware). In the message frame the UnitKey must be 0xFFFFFFFF not just 0xFFFF.
Yes, elapsedMillis was a new one for me too a few weeks ago, but seems useful. But it does result in somewhat different logic in cases like this.
In this particular case I think your version is neater as you have fewer branches to keep track of. Your method looks like it ought to work, and yes, I think you're right, it shouldn't need any resets. I'll need to test it again when I next get a chance - for some reason it wasn't working for me, but it's entirely possible I've got something wrong somewhere else.
Thanks - so far I haven't dug too deeply into the underlying protocol, and have simply made use of your existing methods. I'll have to investigate more thoroughly and give it a go.
Just to keep in mind: It is absolutely essential to increase rx queue size for the MIDI_big_buffer class in "USBHost_t3" : RX_QUEUE_SIZE = 2048 should do the job, i use 4096 to be safe. If you don't, program gets stuck as well. I say this, because i got into this problem again after updating library and forgot to set buffer again!
Thanks - yes, seems to be okay - I've had the same issue before with the library updating.
Hi Martin,
Apologies for the length of this post...
I'm still having trouble with some freezes, and struggling to know where to begin to debug, but perhaps you can help with some suggestions. It seems to work fine to start with, but freezes up a few seconds/10s of seconds/minutes in. I've noticed that your timeout solution seems to work okay, but that a full pedal freeze is often preceded by a short series of these timeouts - more on this later.
I don't seem to have this problem at all if I turn off the expression pedal -> volume adjustment code. But I guess it is possible that the problem always has the potential to happen, but by running a continuous sweep instead of intermittently pressing buttons, the opportunities for clashes are multiplied simply from having a much larger number of comms operations over a given time.
A couple of questions then, to see if this makes any sense to you (everything in points 1-3 below is with the expression pedal code commented out; point 4 has it turned back on again):
Firstly, I've switched to your version of WorkingTimer_Tick(), with the only changes being to add some detail to the Serial debug messages (_id and _time_stamp values).
What sorts of values would you expect to see here?
When first initialising, I get message IDs # 1-27, most of which take a couple of ms to execute the comms - all looks good to me.
When working with the amp settings (i.e. not loaded patches), I get messages like this, which again look sensible to me (apart from the time duration on msg #101...?), for example, toggling Reverb unit using Switch_On_Off_Reverb_Unit():
Button 10 clicked
button_state: 10
Old UI_state: 1
0000 f0 00 01 0c 24 02 4d 00 02 00 00 07 00 0a 00 00
0010 00 10 00 00 00 00 00 00 00 00 00 00 f7
0000 f0 00 01 0c 24 02 4d 00 03 00 00 0f 00 3c 01 00
0010 00 32 01 00 00 00 03 00 00 00 00 00 00 00 00 00
0020 00 00 00 00 f7
Reverb unit switched off
New UI_state: 1
msg #1000 sent out
msg #1000 dequ no ack, no answ, t=0
msg #1001 sent out
THR30_II: 12-Byte-Message Acknowledge for Message #1001
msg #1001 dequ ack, no answ, t=1
THR30_II: 20-Byte-Message (Status) (enum): ready
or changing the reverb type using ReverbSelect() followed by CreatePatch():
==============================
Button 10 held
button_state: 20
Old UI_state: 1
Reverb unit switched from Hall to Spring
Create_patch():
Datalen: 594
Number of slices: 2
Length of last slice: 174
Create_patch(): Ready outsending.
New UI_state: 1
msg #100 sent out
msg #100 dequ no ack, no answ, t=0
msg #101 sent out
msg #101 dequ no ack, no answ, t=2346836497
msg #102 sent out
msg #102 dequ no ack, no answ, t=0
msg #103 sent out
THR30_II: 12-Byte-Message Acknowledge for Message #103
msg #103 dequ ack, no answ, t=101
THR30_II: 24-Byte-Message
Actual user setting was dumped to PC. Following values: 0x2 and 0x0
THR30_II: 20-Byte-Message (Status) (enum): ready
THR30_II: 24-Byte-Message
Parameter change report: GuitProcInputGain 100
THR30_II: 24-Byte-Message
Parameter change report: GuitProcOutputGain 100
Most operations take a few ms, except for operations involving createPatch(), as expected, which take around 90-100ms.
==============================
Button 10 clicked
button_state: 10
Old UI_state: 2
0000 f0 00 01 0c 24 02 4d 00 28 00 00 07 00 0a 00 00
0010 00 10 00 00 00 00 00 00 00 00 00 00 f7
0000 f0 00 01 0c 24 02 4d 00 29 00 00 0f 00 3c 01 00
0010 00 32 01 00 00 00 03 00 00 00 00 00 00 00 00 00
0020 00 00 00 00 f7
Reverb unit switched off
New UI_state: 2
msg #1000 sent out
msg #1000 dequ no ack, no answ, t=0
msg #1001 sent out
THR30_II: 12-Byte-Message Acknowledge for Message #1001
msg #24792 dequ ack, no answ, t=2
THR30_II: 20-Byte-Message (Status) (enum): ready
and
==============================
Button 10 held
button_state: 20
Old UI_state: 2
Reverb unit switched from Spring to Room
Create_patch():
Datalen: 604
Number of slices: 2
Length of last slice: 184
Create_patch(): Ready outsending.
New UI_state: 2
msg #100 sent out
msg #24792 dequ no ack, no answ, t=55
msg #101 sent out
msg #54468 dequ no ack, no answ, t=0
msg #102 sent out
msg #102 dequ no ack, no answ, t=0
msg #103 sent out
THR30_II: 12-Byte-Message Acknowledge for Message #103
msg #103 dequ ack, no answ, t=94
THR30_II: 24-Byte-Message
Actual user setting was dumped to PC. Following values: 0x2 and 0x0
THR30_II: 20-Byte-Message (Status) (enum): ready
THR30_II: 24-Byte-Message
Parameter change report: GuitProcInputGain 100
THR30_II: 24-Byte-Message
Parameter change report: GuitProcOutputGain 100
Is this what you expect to see? If not, any idea what might be causing this?
updatemastervolume(1023) = 100.00
0000 f0 00 01 0c 24 02 4d 00 76 00 00 07 00 0a 00 00
0010 00 10 00 00 00 00 00 00 00 00 00 00 f7
0000 f0 00 01 0c 24 02 4d 00 77 00 00 0f 00 0c 01 00
0010 00 4c 00 00 00 00 04 00 00 00 00 00 40 00 3f 00
0020 00 00 00 00 f7
Timeout waiting for acknowledge. Discarded Message #14560 t=251
msg #1000 sent out
msg #19720 dequ no ack, no answ, t=0
msg #1001 sent out
Timeout waiting for acknowledge. Discarded Message #1001 t=251
msg #1000 sent out
msg #1000 dequ no ack, no answ, t=0
msg #1001 sent out
updatemastervolume(930) = 90.92
0000 f0 00 01 0c 24 02 4d 00 78 00 00 07 00 0a 00 00
0010 00 10 00 00 00 00 00 00 00 00 00 00 f7
0000 f0 00 01 0c 24 02 4d 00 79 00 00 0f 00 0c 01 00
0010 00 4c 00 00 01 00 04 00 00 00 00 40 00 68 3f 00
0020 00 00 00 00 f7
Timeout waiting for acknowledge. Discarded Message #1001 t=269
updatemastervolume(670) = 65.53
0000 f0 00 01 0c 24 02 4d 00 7a 00 00 07 00 0a 00 00
0010 00 10 00 00 00 00 00 00 00 00 00 00 f7
0000 f0 00 01 0c 24 02 4d 00 7b 00 00 0f 00 0c 01 00
0010 00 4c 00 00 01 00 04 00 00 00 00 40 00 27 3f 00
0020 00 00 00 00 f7
msg #1000 sent out
updatemastervolume(305) = 29.88
0000 f0 00 01 0c 24 02 4d 00 7c 00 00 07 00 0a 00 00
0010 00 10 00 00 00 00 00 00 00 00 00 00 f7
0000 f0 00 01 0c 24 02 4d 00 7d 00 00 0f 00 0c 01 00
0010 00 4c 00 00 00 00 04 00 00 00 00 00 40 19 3e 00
0020 00 00 00 00 f7
msg #1000 dequ no ack, no answ, t=3755254863
updatemastervolume(98) = 9.67
0000 f0 00 01 0c 24 02 4d 00 7e 00 00 07 00 0a 00 00
0010 00 10 00 00 00 00 00 00 00 00 00 00 f7
0000 f0 00 01 0c 24 02 4d 00 7f 00 00 0f 00 0c 01 00
0010 00 4c 00 00 00 00 04 00 00 00 00 00 40 46 3d 00
0020 00 00 00 00 f7
msg #1001 sent out
updatemastervolume(11) = 1.17
0000 f0 00 01 0c 24 02 4d 00 00 00 00 07 00 0a 00 00
0010 00 10 00 00 00 00 00 00 00 00 00 00 f7
0000 f0 00 01 0c 24 02 4d 00 01 00 00 0f 00 0c 01 00
0010 00 4c 00 00 00 00 04 00 00 00 00 00 00 40 3c 00
0020 00 00 00 00 f7
updatemastervolume(0) = 0.10
0000 f0 00 01 0c 24 02 4d 00 02 00 00 07 00 0a 00 00
0010 00 10 00 00 00 00 00 00 00 00 00 00 f7
0000 f0 00 01 0c 24 02 4d 00 03 00 00 0f 00 0c 01 00
0010 00 4c 00 00 00 00 04 00 00 00 00 00 00 00 00 00
0020 00 00 00 00 f7
updatemastervolume(138) = 13.57
0000 f0 00 01 0c 24 02 4d 00 04 00 00 07 00 0a 00 00
0010 00 10 00 00 00 00 00 00 00 00 00 00 f7
0000 f0 00 01 0c 24 02 4d 00 05 00 00 0f 00 0c 01 00
0010 00 4c 00 00 00 00 04 00 00 00 00 00 00 0b 3e 00
0020 00 00 00 00 f7
Timeout waiting for acknowledge. Discarded Message #14560 t=291
updatemastervolume(812) = 79.39
0000 f0 00 01 0c 24 02 4d 00 06 00 00 07 00 0a 00 00
0010 00 10 00 00 00 00 00 00 00 00 00 00 f7
0000 f0 00 01 0c 24 02 4d 00 07 00 00 0f 00 0c 01 00
0010 00 4c 00 00 00 00 04 00 00 00 00 40 00 4b 3f 00
0020 00 00 00 00 f7
msg #1000 sent out
msg #1000 dequ no ack, no answ, t=3755255156
msg #1001 sent out
updatemastervolume(936) = 91.50
0000 f0 00 01 0c 24 02 4d 00 08 00 00 07 00 0a 00 00
0010 00 10 00 00 00 00 00 00 00 00 00 00 f7
0000 f0 00 01 0c 24 02 4d 00 09 00 00 0f 00 0c 01 00
0010 00 4c 00 00 00 00 04 00 00 00 00 40 00 6a 3f 00
0020 00 00 00 00 f7
updatemastervolume(1023) = 100.00
0000 f0 00 01 0c 24 02 4d 00 0a 00 00 07 00 0a 00 00
0010 00 10 00 00 00 00 00 00 00 00 00 00 f7
0000 f0 00 01 0c 24 02 4d 00 0b 00 00 0f 00 0c 01 00
0010 00 4c 00 00 00 00 04 00 00 00 00 00 40 00 3f 00
0020 00 00 00 00 f7
Timeout waiting for acknowledge. Discarded Message #19812 t=251
msg #1000 sent out
msg #1000 dequ no ack, no answ, t=3755255407
msg #1001 sent out
Timeout waiting for acknowledge. Discarded Message #1001 t=251
msg #1000 sent out
msg #1000 dequ no ack, no answ, t=2447438450
msg #1001 sent out
Timeout waiting for acknowledge. Discarded Message #49352 t=251
So it seems to recover okay from a few ack timeouts, but as soon as one has happened, it usually isn't long before a few more timeouts in quick succession, followed by a freeze. From a few trials just now, it looks like it might happen consistently after exactly 7 timeouts, which is interesting... Perhaps indicative of a buffer size being exceeded somewhere...?
Throughout all of this, and even once the pedal has frozen, the amp top panel controls work fine, so the amp appears to be all okay. But the pedal won't talk to it again until I restart both the amp and pedal.
Any suggestions gratefully received. Thanks very much, Jo
Hi @martinzw,
A quick update from some further testing:
All the best, Jo
Hi Jo,
to get a bit closer to your problems I added a pedal for GuitarVolume in my own device as well and included it into my software.
I poll the analog value on A0 inside the 150ms GUI-Timer to ensure the pedal not sending it's values too often. Until now, no crash has happend with my code.
Perhaps you can have a look at my code and how i send the parameter change to the THR.
Some guesses, why your code could lead to crashes:
-your button handling and other stuff as well takes more time in the main loop
-perhaps you have done something wrong sending the frame for extended parameter change "GuitarVolume" ? For example coded a wrong value inside this frame?
-try to avoid the use of dynamically allocated memory wherever you can (e.g. avoid String-Type, if possible)
-use TRACE-Macros to be able to disable Serial debug output, that slows things down as well. (I admit, that I do it not everywhere as well, but i would be better). This way you can try, if your app gains stability if less debug output is send out so the Serial monitor.
-I used a RAM-Monitor add on in first phases of development to keep an eye on used heap and so on. It is commented out in the released code: //RamMonitor rm; //During development to keep an eye on stack and heap usage and in setup(): //rm.run(); //keep Memory Monitor up to date (during development to keep an eye on stack and heap usage) if you for example ask "rm.heap_free()" in a slow timer and print it out you can see, i f there are any problems.
Thanks for the useful list of suggestions - I'll work my way through them in due course. On the sample frequency issue, I had been hoping to get the speed up to something a fair bit faster than 150ms (as it makes any sweep into an audible set of jumps rather than a smooth sweep), but maybe that is just unrealistic with this setup.
1) timing Of course you can use a shorter timer period than 150ms. I used it for convenience, because my "tick1" for GUI was already there. But it is important to not doing it in the fast main-loop. So you (the programmer) stay in control of the timing, and not depend on perhaps to many frames in short time occuring from fast pedal moves or jitter. For my part using the 150ms-Timer sounded quite reasonable for me.
2) heap and stack For the RamMonitor i used https://github.com/veonik/Teensy-RAM-Monitor I first got stuck, when I re activated it for testing yesterday. But i noticed, that I forgot "rm.Initialize()" in the setup()
So you should use :
RamMonitor rm; //Create Monitor Object in global namespace
setup() {... rm.Initialize(); ...}
and to read out results (e.g. in 3500ms Timer tick4()) use:
rm.run(); //keep Memory Monitor up to date (during development to keep an eye on stack and heap usage)
TRACE_V_THR30IIPEDAL(printf("Used Heap %lu bytes. ",rm.heap_used());)
TRACE_V_THR30IIPEDAL(printf(" Used Stack %lu bytes.\r\n",rm.stack_used());)
the methods rm.warning_lowmem() and rmwarning_crash()
could be helpful as well.
The results in my program were: Used Heap 142800 bytes. Used Stack 168 bytes. And this did not grew up very much while using VolumePedal and sending patches.
Edit: rammon is for Teensy 3.x only, not for 4.1. I don't know, if there is something like this for 4.1
3) "on_ack_queue" I found and remembered, that i had already begun to implement a mechanism to follow a workload package, kind of "action pack". This is useful, if messages must not simply be sent out, but have to be followed by other stuff to do. For example a flag has to be set or some more frames have to be sent out, after the first message, that we had sent out to THR , was acknowledged. By now in my program I do not make extensive use of that mechanism. But for mor complex jobs, that our pedals have to do, we will have to use a concept like this - it is a matter of following the (not well known) communications protocol between host and THR.
Hi Martin, thanks very much for the suggestions. (Sorry for the very slow reply - sometime real life gets in the way of projects for a while!)
I hadn’t tried with guitarVolume yet, and for now I was using THR_Values.SetControl(CTRL_MASTER, scaledvolume); to set the amp master volume. I need to think through what I actually want here, as I can’t actually think of a situation where I would suddenly want to turn up the guitarVolume to 100. Ideally I’d set the max volume (at the start of a gig, say) using the knob on the top panel, then have the pedal sweep between 0 and that value. So perhaps read in the knob value, then set the outgoing guitarVolume to [knob value] x [a multiplier value from the pedal sweep].
I will keep looking into this, but in the meantime the pedal works with no issues when the external pedals are detached, and there are plenty of other features for me to get on with in the meantime.
Thanks, Jo
Hi Martin, I wonder if you can offer any suggestions on this one:
I have been trying, in my adaptation of your code, to make use of an expression pedal input to control a given parameter - something you suggest yourself in your description of the protocol. Have you ever tried doing this yourself?
I have set up a function which polls the pedal input (at a chosen sample rate, and with a delta threshold so that it only triggers when an actual change has happened, rather than sending a constant stream of updates) and uses this to send an update to the amp using the SetControl() method - in this case changing the amp master control, but could be any parameter.
Single changes seem to work fine. Multiple/continuous changes seem to work fine... for a while. I am running into an issue where it runs fine for a few seconds, or a few 10s of seconds (and sweeps perfectly well across the full range of values 0-99), but then at some indeterminate point, the amp appears to just stop accepting the messages. It looks like the pedal is still generating the messages, but I am not sure if it is still sending them correctly, and the amp no longer responds until I reset everything by power cycling the amp and pedal. (Direct changes via the amp's top panel all work as normal, and still send update messages to the pedal, which show up on the Serial Monitor, so I fairly confident the amp is still behaving correctly.)
When this happens, apart from pedal's button inputs no longer affecting the amp, the obvious difference in the Serial Monitor output is that the messages from WorkingTimer_Tick() (e.g. "sent out", "dequ no ack, no answ", etc) and ParseSysEx() (e.g. "THR30_II: 12-Byte-Message Acknowledge for Message #1001") stop. This can be seen in the code block below (where the 1/2-digit numbers are the values I am sending).
Have you tried this at all? Have you seen any similar behaviour? Any suggestions as to the cause, or what I can do to fix (or avoid) it?
Any suggestions appreciated. Thanks, Jo