ThreadDebug Lib - Unresponsive After Start

[jwestmoreland]

Hello All,

Originally discussed here: https://github.com/arduino/ArduinoCore-mbed/issues/16

I've tried using the ThreadDebug Library with the Portanta_M7 - at the top of my sketch:

include

// UartDebugCommInterface debugComm(SERIAL1_TX, SERIAL1_RX, 230400); // ThreadDebug threadDebug(&debugComm, DEBUG_BREAK_IN_SETUP);

UsbDebugCommInterface debugComm(&SerialUSB); ThreadDebug threadDebug(&debugComm, DEBUG_BREAK_IN_SETUP);

---

The debug console window prints: There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "--host=i686-w64-mingw32 --target=arm-none-eabi". Type "show configuration" for configuration details. For bug reporting instructions, please see: . Find the GDB manual and other documentation resources online at: . For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from c:/temp/Display/Display.arduinobeta.mbed.envie_m7.elf... done. GDB unhandled notify: cmd-param-changed: {"param":"pagination","value":"off"} GDB unhandled notify: cmd-param-changed: {"param":"target-charset","value":"ASCII"} Remote debugging using COM4

If I type show configuration it says: Please start a debug session to evaluate - but a debug session is running.

---

Another session where it just disconnects after starting: . Find the GDB manual and other documentation resources online at: . For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from c:/temp/Display/Display.arduinobeta.mbed.envie_m7.elf... done. GDB unhandled notify: cmd-param-changed: {"param":"pagination","value":"off"} GDB unhandled notify: cmd-param-changed: {"param":"target-charset","value":"ASCII"} Remote debugging using COM4 setup () at c:\temp\Display/Display.ino:214 214 { Will reset on next continue. Note: automatically using hardware breakpoints for read-only addresses. Remote connection closed

So, any help appreciated.

Thanks, John W.

[adamgreen]

My experience with using ThreadDebug is with standalone GDB so it might take me some time to get up to speed on how to run it from the Pro IDE.

The screen shot you show above looks like it is working to me. I do notice that there is some code shown in the IDE above inside of an #ifdef block that uses Serial.println() calls. It looks like you have ThreadDebug configured to use the same USB based Serial connection which would definitely confuse GDB and/or ThreadDebug. ThreadDebug should be configured as the sole user of its serial connection to GDB. Maybe you could switch one of them to Serial1 and see if that helps?

[jwestmoreland]

@adamgreen ,

I switched to Serial1 and dedicated the debugger there - getting this now:

and "show warranty" for details. This GDB was configured as "--host=i686-w64-mingw32 --target=arm-none-eabi". Type "show configuration" for configuration details. For bug reporting instructions, please see: . Find the GDB manual and other documentation resources online at: . For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from c:/temp/Display/Display.arduinobeta.mbed.envie_m7.elf... done. GDB unhandled notify: cmd-param-changed: {"param":"pagination","value":"off"} GDB unhandled notify: cmd-param-changed: {"param":"target-charset","value":"ASCII"} Remote debugging using COM9 Ignoring packet error, continuing... warning: unrecognized item "timeout" in "qSupported" response Ignoring packet error, continuing... Remote replied unexpectedly to 'vMustReplyEmpty': timeout "monitor" command not supported by this target.

Do you have the Portenta working with ThreadLib and the standalone GDB?

Thanks, John

[adamgreen]

Those errors make it look like GDB can't successfully communicate with ThreadDebug over COM9. Is it possible that the baud rates don't match between GDB and ThreadDebug?

Do you have the Portenta working with ThreadLib and the standalone GDB?

Yes, I have used it that way before. This is the command line I use on macOS and might need to be modified a bit to work in your Windows environment:

arm-none-eabi-gdb --baud 230400 -ex "set target-charset ASCII" -ex "set print pretty on" -ex "set remotelogfile mri.log" -ex "target remote /dev/tty.usbserial-FTDXR25Z" BinaryFilename.elf

I use the remotelogfile parameter to allow diagnosing any issues that GDB has in communicating with ThreadDebug.

[jwestmoreland]

This is working from the command line:

C:\Users\John\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\arm-none-eabi-gdb -ex "set pagination off" --baud 230400 -ex "set target-charset ASCII" -ex "target remote COM4" Display.arduinobeta.mbed.envie_m7.elf

Still not working in IDE.

[jwestmoreland]

Hello All,

I've tried this on another computer - fresh installation and all - and am getting this now:

C:\tmp\Display>C:\Users\John\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\arm-none-eabi-gdb -ex "set pagination off" --baud 230400 -ex "set target-charset ASCII" -ex "set print pretty on" -ex "set remotelogfile mri.log" -ex "target remote COM11" Display.arduinobeta.mbed.envie_m7.elf GNU gdb (GNU Tools for Arm Embedded Processors 7-2017-q4-major) 8.0.50.20171128-git Copyright (C) 2017 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "--host=i686-w64-mingw32 --target=arm-none-eabi". Type "show configuration" for configuration details. For bug reporting instructions, please see: http://www.gnu.org/software/gdb/bugs/. Find the GDB manual and other documentation resources online at: http://www.gnu.org/software/gdb/documentation/. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from Display.arduinobeta.mbed.envie_m7.elf...done. COM11: No such file or directory. (gdb)

And am getting the 'No such file or directory error'. I've made sure that as much as possible is in the path - and even did a complete uninstall/reinstall - and still get the No such file or directory error.

Note: I get the same error whether in the IDE or running from the command line.

Anyone know the root cause of this?

Thanks! John

[jwestmoreland]

OK - I changed COM11 to COM2 and it appears to have fixed that...on the CLI...

C:\tmp\Display>C:\Users\John\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\arm-none-eabi-gdb -ex "set pagination off" --baud 230400 -ex "set target-charset ASCII" -ex "set print pretty on" -ex "set remotelogfile mri.log" -ex "target remote COM2" Display.arduinobeta.mbed.envie_m7.elf GNU gdb (GNU Tools for Arm Embedded Processors 7-2017-q4-major) 8.0.50.20171128-git Copyright (C) 2017 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "--host=i686-w64-mingw32 --target=arm-none-eabi". Type "show configuration" for configuration details. For bug reporting instructions, please see: http://www.gnu.org/software/gdb/bugs/. Find the GDB manual and other documentation resources online at: http://www.gnu.org/software/gdb/documentation/. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from Display.arduinobeta.mbed.envie_m7.elf...done. Remote debugging using COM2 setup () at c:\tmp\Display/Display.ino:218 218 { (gdb) finish Run till exit from #0 setup () at c:\tmp\Display/Display.ino:218 Note: automatically using hardware breakpoints for read-only addresses. [New Thread 604069160] main () at C:\Users\john\AppData\Local\Arduino15\packages\arduinobeta\hardware\mbed\1.2.1\cores\arduino\main.cpp:47 47 C:\Users\john\AppData\Local\Arduino15\packages\arduinobeta\hardware\mbed\1.2.1\cores\arduino\main.cpp: No such file or directory. (gdb) continue Continuing.

after adding to my path... [New Thread 604069160] main () at C:\Users\john\AppData\Local\Arduino15\packages\arduinobeta\hardware\mbed\1.2.1\cores\arduino\main.cpp:47 47 loop(); (gdb) continue Continuing.

[adamgreen]

I know that in older versions of Windows, you needed to prepend higher COM port numbers with "\.\". For example:

\\.\COM11

[jwestmoreland]

Hmmm - I'm running the latest version (test mode) of Win10...

Have you been able to get the debugger running in the new IDE? It will still disconnect when I try to start it.

[adamgreen]

Hmmm - I'm running the latest version (test mode) of Win10...

I suspect that it is true for Windows 10 as well.

Have you been able to get the debugger running in the new IDE?

Sorry, I haven't had time to switch gears back to this debugger project yet. Just trying to finish up my current project first so that I can dedicate more time to looking into it.

[jwestmoreland]

Have you run OpenOCD on this target yet?

[adamgreen]

I have used Segger's JLink hardware and software but not OpenOCD.

[jwestmoreland]

I have a Segger handy - so maybe I'll try that.

[jwestmoreland]

Adam,

Another interesting tidbit I've discovered - if you use a hub that isn't delivering enough power - you can get the following error(s):

C:\tmp\Display>C:\Users\John\AppData\Local\Arduino15\packages\arduino\tools\arm-none-eabi-gcc\7-2017q4\bin\arm-none-eabi-gdb -ex "set pagination off" --baud 230400 -ex "set target-charset ASCII" -ex "set print pretty on" -ex "set remotelogfile mri.log" -ex "target remote COM2" Display.arduinobeta.mbed.envie_m7.elf GNU gdb (GNU Tools for Arm Embedded Processors 7-2017-q4-major) 8.0.50.20171128-git Copyright (C) 2017 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "--host=i686-w64-mingw32 --target=arm-none-eabi". Type "show configuration" for configuration details. For bug reporting instructions, please see: http://www.gnu.org/software/gdb/bugs/. Find the GDB manual and other documentation resources online at: http://www.gnu.org/software/gdb/documentation/. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from Display.arduinobeta.mbed.envie_m7.elf...done. Remote debugging using COM2 setup () at c:\tmp\Display/Display.ino:217 217 { (gdb) finish Run till exit from #0 setup () at c:\tmp\Display/Display.ino:217 Note: automatically using hardware breakpoints for read-only addresses. warning: Error removing breakpoint 0 warning: Error removing breakpoint -5 warning: Error removing breakpoint -5 Remote connection closed (gdb) quit

I guess in the core code those could be low power or 'brown-out' warnings maybe.

FWIW, John

[jwestmoreland]

@adamgreen - Hello Adam,

I was wondering - is there a setting for ThreadLib that makes it work with JLink's GDB server? I've been trying to get that working - it runs (sortof) but when adding debugBreak()'s in the code - seems to be ignoring that.

-ex "target remote localhost:2331" appears to work fine, but the behavior is different.

Thanks In Advance, John

[adamgreen]

No, ThreadLib itself is a GDBServer that runs on the Portenta and allows GDB to connect via the serial port. Either ThreadDebug should be used or a hardware SWD debugger (ie. JLink). Not both.

[jwestmoreland]

Adam,

That's what I thought and wanted to ask to be sure.

It's nice to have a hardware connection to the SWD port - maybe as an enhancement it could be possible to add that to ThreadLib? Maybe jlink.cfg for ThreadLib - JLink as the I/F only of course?

Thanks, John

[adamgreen]

It's nice to have a hardware connection to the SWD port - maybe as an enhancement it could be possible to add that to ThreadLib? Maybe jlink.cfg for ThreadLib - JLink as the I/F only of course?

I am not quite sure what you mean. Can you provide more details of what are you thinking?

[jwestmoreland]

Adam,

Just a thought/request of being able to possibly use ThreadLib via an interface that interacts directly with SWD vs. the UART(s) or USB - if that's even possible with ThreadLib. JLink, as you know, is just one possible interface for that.

Thanks, John

[adamgreen]

I don't know that ThreadDebug could be modified to do that and even if it could, I don't know that there would really be any advantage to supporting it. If you are going to use SWD debugging, then the best experience will be connecting the SWD debugger module directly to the hardware SWD pins.

ThreadDebug is primarily a GDB debug stub, so using it from another GDB debug stub would completely waste most of the code contained within ThreadDebug.

Some JTAG/SWD debug stubs (JLink included I) do support use of the debug monitor interrupt handler to debug low priority code while allowing high priority code to continue running but I don't think there is a standard for that and would require a separate implementation for each one. This page describes Segger's support of that feature. That wouldn't be a feature that I would want to add to ThreadDebug but someone else might be able to figure out how to build another library that would support it.

[jwestmoreland]

@adamgreen ,

I think the possible enhancement with the Segger I/F you pointed out

https://www.segger.com/products/debug-probes/j-link/technology/monitor-mode-debugging/

sounds like an interesting enhancement.

Thanks, John

[adamgreen]

Yeah it is a cool feature but would require a completely different library, not ThreadDebug.

ThreadDebug already solves the problem of keeping high priority interrupts/threads running while debugging in its own way. It is even higher level than what JLink supports since ThreadDebug has knowledge of mbed-os's thread scheduler.

[jwestmoreland]

Adam,

OK - I'm eagerly awaiting ANY documentation on ThreadLib... <Oops - guess I need to look here: MRI (Monitor for Remote Inspection)> -

I think we can agree these would be interesting enhancements:

Target application has a motor control part which makes use of Pulse-width modulation (PWM), and the PWM units also stop when the CPU is halted, possibly leaving the PWM unit and motor control in an undefined or even dangerous state. Monitor mode debugging allows the PWM part of the application to keep running while the rest of the application is halted and can be debugged. Target application has a Bluetooth part where halting the complete target application would cause the Bluetooth communication to break down. Monitor mode debugging allows the Bluetooth "keep-alive" interrupt(s) to run and handle the low-level Bluetooth communication while the rest of the application is halted and can be debugged. Target application has an Ethernet part where halting the CPU would cause the IP stack etc. to become totally unresponsive, so packets cannot be received anymore, even keep-alive packets etc. Monitor mode debugging allows the developer to debug in this scenario, too. Target application has a CAN bus part where halting the CPU would cause messages to get lost. In monitor mode, the CPU is able to handle incoming messages while the main application is halted.

Thanks, John

[jwestmoreland]

Well, to get back on topic - this is still happening:

Disconnects.

I'd like to mention something else as well I'm seeing - and I don't have a great explanation yet - but I can change the COM port setting to be something else - and when I compile with the IDE - I'll get a DFU error; but if I change it - it'll work.
I don't know if this has anything to do with com port #'s being greater than 10 or anything like that - but it's a little strange and I'd like to be able to explain this. I was using the 1.8.12/13 IDE to compile - it doesn't seem to have this issue with the COM port number (under win10).

Thanks, John

[adamgreen]

@jwestmoreland Did you have to jump through some special hoops to get the Pro IDE working with the Portenta H7 like documented in this post? I didn't even get things to build tonight, let alone run under the debugger.

[jwestmoreland]

@adamgreen , No, not really. I followed those instructions and was able to get some examples to build. Now, one issue I did have was that I had made the mistake of letting M$ install OneDrive on a few of my machines. That 'tool', much like my experience with Carbonite, successfully mangled my dev environment beyond recognition; so, I uninstalled the Arduino tools, removed M$ OneDrive, reinstalled; made sure my 'temp' paths were all AOK, then proceeded with the board manager install. The one issue right now is arduino-beta won't work in the path - you have to change that to arduinobeta - so, when I use 1.8.13 now, I change it to arduino-beta, and then using the Pro IDE arduinobeta - but that's covered in the post - any tool like OneDrive or Cabonite could be wreaking havoc if you have that running in the background.

HTH, John

[jwestmoreland]

Adam,

Make sure you've fixed up the API directory under:

C:\Users\\AppData\Local\Arduino15\packages\arduino-beta\hardware\mbed\1.2.1\cores\arduino

There could just be a file there vs. a directory depending on how you did the install.

I think it's OK just to grab the latest from here in branch namespace_arduino :

git clone git@github.com:arduino/ArduinoCore-API -b namespace_arduino

If you're running on windows - otherwise I think you need to fix the symbolic link.

If that's not correct the IDE can throw a lot of errors.

Regards, John

[jwestmoreland]

Adam,

One more thing - and this one just happened - but, I backed up to a previous branch to check something -

Check in your environment where any TMP or TEMP variable points - on the machine I'm using now:

TMP and TEMP are both set to c:\AppData\Local\Temp

I just had a compile fail giving an error about the WiFi Class - it also pointed to the dir that was in TMP.

So, I had to clean TMP manually of all arduino-XXX... temp sketch directories - so, that's another issue that could be happening - make sure you don't have any precompiled stuff in TMP or TEMP that the compiler may grab that's outdated.

HTH, John

[adamgreen]

Looks like ThreadDebug is working from GDB CLI directly so there appears to be no issue with ThreadDebug itself.

I will experiment with the Pro IDE again once it stabilizes a bit more in the future. I don't feel like jumping through a bunch of hoops just to get it compiling and uploading before I can even try using the debugger.

[jwestmoreland]

Adam,

OK - I just tried it again - but it does the same thing as I've reported above - it'll start then it will disconnect.

I'm looking forward to running this in the new Pro IDE - I'd like to keep this thread open if that's OK with everyone.

Regards, John

[adamgreen]

In one of your screenshots above, I see the following text: "Will reset on next continue" which means that a "monitor reset" command is being sent to ThreadDebug which would cause the USB connection to be reset on the next "continue" command so that is probably what is happening.

[jwestmoreland]

Adam,

OK - how do I prevent that from happening? Editing the platform.txt now...

I've also seen messages that say 'monitor isn't enabled/available for your target' also - at least a message similar to that.

Regards, John

[jwestmoreland]

@adamgreen ,

Hello Adam - are you still looking at this?

0.1.2 of the IDE is out - just wondering.

@kittaakos is looking at this now as well - but from the IDE side.

Regards, John

[adamgreen]

Hello Adam - are you still looking at this?

0.1.2 of the IDE is out - just wondering.

@jwestmoreland Does the latest version of the IDE support the Portenta H7 right out of the gate or do users still need to jump through hoops to get it working like before?

[jwestmoreland]

Hello Adam - are you still looking at this? 0.1.2 of the IDE is out - just wondering.

@jwestmoreland Does the latest version of the IDE support the Portenta H7 right out of the gate or do users still need to jump through hoops to get it working like before?

Hello Adam,

IDE appears to be 'better' now.

Regards, John W.