search

ARMmbed / DAPLink

https://daplink.io
Apache License 2.0
2.31k stars 976 forks source link

DAPLink/CMSIS-DAP & Debug Adapter Protocol #560

Closed 0Grit closed 5 years ago

0Grit commented 5 years ago

Context: Debug Adapter Protocol

Does anyone have insight as to how a Debug Adapter Protocol compliant "Debug Adapter" for DAPLink might look?

Would GDB and or pyOCD still be involved? Or would the Debug Adapter connect directly to DAPLink?

flit commented 5 years ago

Interesting, I hadn't seen this protocol before! There are multiple ways one could go about supporting it. You could create a GDB/MI translator in order to continue using the existing gdb infrastructure. However, this probably is a waste of time since VSCode already has full support for GDB/MI (unless you are wanting to use another IDE).

The Debug Adapter Protocol doesn't have any impact on DAPLink, which is simply a low level wire protocol interface for SWD/JTAG and UART.

Ultimately, it's probably best handled by pyOCD as an additional protocol. But pyOCD would need much more complete built-in debug capabilities, such as understanding the type hierarchy in DWARF debug info. I do want to get there eventually, but it will take some time. 😄 I've added mbedmicro/pyOCD#488 to track the requirement in pyOCD.

0Grit commented 5 years ago

@flit So,

This following questions might be better spent in the pyOCD repo but maybe I can sneak them in,

cederom commented 5 years ago

This so called "Debug Adapter Protocol" [dap] that seems to be a Microsoft invention, sorry to say but it was probably architected by someone with a little knowledge and/or experience in hardware design and debugging, and you can think that way just in the first second because abbreviation DAP is already taken and commonly used by ARM cores for Debug Access Port used by JTAG and SWD transport as well as debug blocks and related commands transported.. so putting another DAP here would only lead to another level of confusion.. as some folks may really like it.. I would strongly suggest using something like MOSAD (Microsoft Open Source Adapter Debugger), or at least DAP-MS (Debug Adapter Protocol by MicroSoft), or anything else just not existing names and abbreviations from the same area..

@loverdeg-ep that DAP could behave just as GDB protocol, pyOCD would be the "dapserver" instead "gdbserver", in short words:

Debug Adapter Protocol would work just like GNU Debugger Protocol, as presented on picture below, implemented in [pyocd] project and @flit would be best person to talk to.. I just wonder what are the additional benefits of creating alternative from-scratch DAP solution instead improving existing Open-Source GDB tools ;-)


          MS Windows
           Debugger

  +-------------------------> Debug Adapter Protocol API

          pyOCD BOX
    Debug Adapter Protocol

  +-------------------------> CMSIS-DAP

           DAPLink
          CMSIS-DAP

  +-------------------------> JTAG/SWD Debug Access Port

         TARGET MCU
      Debug Access Port

[dap] https://github.com/Microsoft/debug-adapter-protocol [gdbserver] https://en.wikipedia.org/wiki/Gdbserver [pyocd] https://github.com/mbedmicro/pyOCD

0Grit commented 5 years ago

@cederom Agree, unfortunately the name is too generic.

My hope is that if it becomes an accepted standard it would eventually provide a specified alternative to GDB's MI. Such that various IDE's and workspace servers (VS Code, Theia, Eclipse Che) can all use the same "Debug Adapters"

Not sure of feasibility, considering resource limitations of currently supported hardware, but my big wish is to have the "debug agent"/"debug adapter" running on the debug probe along with DAPLink. If DAprotocol could be served over TCP/IP then we'd be able remote debug with nothing more than a cortex-m based probe and web based IDE.

flit commented 5 years ago

There is a lot of room for improvement of both GDB/MI and the GDB Remote Serial Protocol (RSP).

To answer your questions, @loverdeg-ep, pyOCD is a Python package that provides host control of Cortex-M devices. A major feature is the gdbserver, but it can also be used for inspecting and controlling devices either programmatically via its Python API or interactively via the REPL interface. pyOCD does not provide an interface for other IDEs, because it doesn't have full debug capabilities. It currently is based on the assumption that gdb (or lldb) will be used as the debugger, and IDEs will connect to gdb via GDB/MI.

For me, it makes a lot of sense to expand pyOCD's capabilities to become a complete debugger and support Microsoft's DAP as a remote protocol for those capabilities, in addition to providing a Python API. For one, gdb has a lot of limitations when it comes to things like multiple stacks (MSP, PSP), secure vs nonsecure worlds, and so on. MS-DAP will probably need enhancements to handle these features too, but it's starting from a much better place than gdb and is a clean protocol specification rather than code.

cederom commented 5 years ago

Why would you like to replace the GDB @loverdeg-ep ? Lots of IDE/SDK already support GDB + GDBServer as Debug backend.. with an OpenOCD project folks did put JTAG + GDBServer + TCP on a single embedded board.. @arekzaluski is working on WebDAP that would allow to control Target MCU using DAPLink over a Web Browser.. did you research well the available options first? :-)

Is GDB really that bad it needs to be replaced by MS-DAP @flit ? No improvements can be done to GDB itself? Is it so obsolete by design?

I guess what MS-DAP can do best is to clean-up and clarify Embedded Debug Architecture and Documentation making it well organised and easy to understand.. thus take over the lead in this area.. some big company finally noted the market fit ;-)

0Grit commented 5 years ago

@cederom Hmm, my surface level research only indicated that @arekzaluski's efforts were directed towards using WebUSB to access a DAPLink probe via a browser. I assume that is specifically for enabling localhost debugging via usb and browser based IDE/debug tools. I don't see any documentation or direct support for exposing a WebUSB DAPLink device to a local network.

I haven't seen any repositories mentioning WebDAP and I'd rather skip USB and go straight to the network if possible. My assumption has been that there is no direct open source solution for this now, trying to get a feel for feasibility and express interest.

@flit thanks for the details! This can be closed or continued at your leisure.

0Grit commented 5 years ago

Before I close this. @thegecko do you have any thoughts on this?

thegecko commented 5 years ago

Thanks for the @mention.

DAP is something we have researched a lot for the Mbed Studio project and it is used in our debug stack to drive gdb/pyOCD.

There seems to be a lot of love for the debug adapter protocol and although developed as part of VSCode, it's also gaining support in other IDEs (e.g. Eclipse, Che and Theia) in a similar way to the language server protocol.

To comment on some of the questions/claims above:

vscode already has has full support for GDB/MI

This is actually via debug adapters using the DAP protocol. The main one for VSCode lives here and another one has recently been created for eclipse CDT here

pyOCD could eventually evolve to provide a "Debug Adapter" "server" in order to provide debugging to any IDE that supports DAProtocol

This would be an awesome addition to pyOCD and remove the need for gdb client/server in a local debug environment. For Mbed Studio, debug commands go through a massive translation stack increasing complexity and latency (Mbed Studio <> DAP <> MI <> GDB Client <> PyOCD GDBServer <> CMSIS-DAP). However, I would think that remote debug scenarios should still use gdb.

I just wonder what are the additional benefits of creating alternative from-scratch

As mentioned, it would massively reduce complexity and the need to rely on gdb in the case of Mbed Studio. It also simplifies any web-based debug solution we may implement as most of the debug adapter implementations can be made to work in-browser.

it makes a lot of sense to expand pyOCD's capabilities to become a complete debugger and support Microsoft's DAP

Agreed

working on WebDAP that would allow to control Target MCU using DAPLink over a Web Browser

You mean DAPjs. This was developed with two primary goals; to flash binaries from a browser and to enable an in-browser debug solution in future. Wiring DAP with DAPjs to accomplish this could be a way to go, but would be a massive task.

flit commented 5 years ago

@thegecko I'm glad we're aligned here! My only comment is that remote debug shouldn't need to use gdb any more than local debug (assuming pyOCD implements MS-DAP). A related future feature addition to pyOCD is to be able to serve a local connection to any supported debug probe across the network, and of course support such a remote debug probe as a client.

0Grit commented 5 years ago

@flit other than it being a python script, what is that 'sauce' that would prevent pyOCD features being implemented on say an STM32 M7?

My thinking is that If i'm already forced to have an MCU next to my target to run DAPLink why not just make that MCU connect to to the network and provide a 'MS-DAP' server directly to my web based IDE.

Taking a 2nd look at my statement above though makes me realize some of the non-trivial things that need to happen first.

What is ARM's current CI/test farm setup? Cheap single board computers + lots of USB hubs?

0Grit commented 5 years ago

@flit any answer for the question above? Very curious to know where all the USB cables go.

flit commented 5 years ago

@loverdeg-ep, sorry I missed your question.

Yes, it's quite non-trivial to get there, but it's totally possible to run the whole debugger on a relatively powerful MCU connected directly to the target. The main issue is really the debugger (MS-DAP) implementation itself.

You could achieve something close to this today by running gdb + pyOCD + Microsoft's GDB/MI to DAP adapter all on a Cortex-A-based MCU such as the i.MX 7D. You could either run DAPLink ported to the i.MX 7D's CM4 core, replacing USB with IPC, plus a special IPC debug probe for pyOCD. Or implement a Linux sysfs GPIO based SWD debug probe for pyOCD. (Either way would be really cool!) The only remotely visible service would be MS-DAP. Of course, that involves a ton of overhead internally.

The main reasons not to go this route are simply price and practicality. It's much cheaper to have a small sub-$2 MCU with a USB connection to provide the host connection to SWD. You already have a high-powered host PC in place, so you might as well use that to run the more complex parts of the debug chain. Managing configuration, software updates, etc is probably easier with a process running on your host PC than if it were on the debug MCU, especially if it's not running Linux.

For our DAPLink and pyOCD CI system, we use Jenkins to control Mac, Linux, and Windows machines. The boards are connected to rack-mounted USB hubs. We can't use single-board computers because we need to test both projects with all three host OSes.

0Grit commented 5 years ago

Managing configuration, software updates, etc is probably easier with a process running on your host PC than if it were on the debug MCU, especially if it's not running Linux.

Easier from a software implementation standpoint for sure. My hope would be that eventually daplink being ported to Mbed OS would ease implementation just enough to make it not an insane effort to bring an "MS-DAP CMSIS-DAP" implementation over as well.

However I also must admit my understanding of what pyOCD is actually doing and how much overhead would come from MS-DAP implementation is near 0.

Thanks for the info on the farm setup.

flit commented 5 years ago

Closing issue. MS-DAP is out of scope for DAPLink as it requires a full debugger. Any further discussion should happen on mbedmicro/pyOCD#488.