Support CDT standalone debugger application

[flit]

CDT supports a standalone debugger application that allows one to debug executables without have to create projects or use the full Eclipse. This mode of use is particularly suitable for embedded projects since developers often use command line build systems and only need Eclipse Embedded CDT for the debugger.

Probably the best approach would be to build an alternate version of the standalone debugger application that includes the Embed CDT plugins. The standalone debugger presents a special UI for native debug that allows users to skip traditional launch config creation (though you can open the standard launch config editor instead). It's not clear whether this is a good idea for Embed CDT since the launch config interfaces are quite a bit more complex than for native debugging.

[ilg-ul]

Yes, this is a tricky one. When the Embed CDT debug plug-ins were created, I assumed they will always run with a full project available. Later on there were some changes intended to make them standalone, but I don't know how successful they were; and personally I don't remember actively using them in this configuration.

As for adding a new standalone debugger... I don't know how much work would require, personally I have very few resources available.

[flit]

I'm pretty sure a project is still required by the plugins in some cases. I quickly tried using it without a project a while ago without much luck. But I think the standalone debugger creates a hidden project for you, so it might not be an issue.

Yeah, I don't have much time to dedicate to it either. Though I do think it would probably increase the visibility and usability of Embed CDT for (the many) folks that don't otherwise use Eclipse. Anyway, it's on my wish list for when I suddenly have a bunch of free time (🤣).

[ilg-ul]

I do think it would probably increase the visibility and usability of Embed CDT for (the many) folks that don't otherwise use Eclipse.

Yes, sure. But there is also an Arabic saying 'If the mountain will not come to Muhammad, then Muhammad must go to the mountain'. :-)

I'm currently working on a version of the managed build process that runs outside Eclipse, and will use CMake/meson/xPack Build, that I'll try to integrate both in Eclipse and in Visual Studio Code. Once this is done, the next step would be to move the functionality of the debug plug-ins (J-Link/OpenOCD/QEMU) to Code.

If you are interested in details, I'd be happy to discuss them with you via email.

[flit]

Sounds very cool! I'm definitely interested in supporting Code; working with you makes a lot of sense. (I've been considering contributing to Marus' cortex-debug plugin since it already supports pyocd but pretty poorly.)

Btw, you really should take a look at pyocd. 😄 It does pretty much everything OpenOCD does (for Cortex-M), except generally easier to use, supports almost every Cortex-M device via builtin support + CMSIS-Packs, and support CMSIS-DAP, STLink, and J-Link.

[ilg-ul]

you really should take a look at pyocd

If you confirm that it runs fine on macOS & Berries, and I can install it somewhere in my home folder, I will give it another try. If I remember right, when you first contributed the plug-in, I tried it, but something went south on my Mac and could not make it work.

BTW, I don't use OpenOCD at all, if I really need hardware debug (since otherwise most of my development is done on QEMU), then I use J-Link.

[flit]

I develop and use pyocd primarily on macOS, my main OS, so yes it's very well supported. 😄 If you do have trouble please do let me know. (And it has improved massively since I first wrote the plugin almost 5 years ago.)

It also works well on RPi, but there are complications with installing due to cmsis-pack-manager (it's backend is written in Rust + the Rust compiler can run out of memory on an RPi + PyPI doesn't support AArch32 wheels + piwheels.org doesn't support building with Rust yet + python packages don't support optional dependencies == much frustration). On my todo list is documentation and improvements to RPi installation.

I agree that J-Link is great. I used to use it almost exclusively, but then I got involved in CMSIS-DAP and the DAPLink project, partially due to work.

[ilg-ul]

I tried to follow the instructions from https://github.com/pyocd/pyOCD/blob/master/README.md on my Mac, where I have both Python 2 & 3 installed, but there's no pip:

ilg@wks ~ % pip install -U pyocd
zsh: command not found: pip
ilg@wks ~ % python --version
Python 2.7.16
ilg@wks ~ % python3 --version
Python 3.7.6

There are some good reasons why I distribute fully standalone binary xPacks.

[flit]

Weird! pip is installed by default with both the official Python installer and via homebrew and most other Python distributions. It may just be a path issue, so try python3 -mpip install -U pyocd. (I might change the install instructions to use this form.)

Another alternative that should always be available in a Python install is easy_install. To get pip run easy_install pip.

(You still need Node.js and npm installed first to use xPacks. 😉)

[ilg-ul]

The line you provided failed with:

ERROR: Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pycparser'
Consider using the `--user` option or check the permissions.

A second attempt with python3 -mpip install -U pyocd --user did the trick:

ilg@wks ~ % python3 -mpip install -U pyocd --user
Collecting pyocd
...
Successfully installed appdirs-1.4.4 cffi-1.14.4 cmsis-pack-manager-0.2.10 colorama-0.4.4 future-0.18.2 hidapi-0.10.1 intelhex-2.3.0 intervaltree-3.1.0 milksnake-0.1.5 naturalsort-1.5.1 prettytable-2.0.0 psutil-5.8.0 pycparser-2.20 pyelftools-0.27 pylink-square-0.8.1 pyocd-0.28.3 pyusb-1.1.0 pyyaml-5.3.1 sortedcontainers-2.3.0 wcwidth-0.2.5
WARNING: You are using pip version 19.2.3, however version 20.3.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
ilg@wks ~ % pyocd --version
0.28.3
ilg@wks ~ % which pyocd
/Users/ilg/Library/Python/3.7/bin/pyocd

You still need Node.js and npm installed first to use xPacks.

Binary xPacks are plain archives, you download them and extract in any folder, and without any special system prerequisites, they just work. You can hardly beat this with any Python application.

It is true that for a full xPacks experience, you need Node.js. But if you read the install documentation carefully and follow the instructions by the book, you always get a functional Node.js/npm environment, on all supported platforms. And once you do this, xpm does the rest of the job.

I don't want to say that Python is not ok, just that, for me, the install experience was not great; you can probably improve it with a better documentation.

[flit]

Sorry you had such trouble! Normally you should never need to run as root or be forced to install for user on macOS. But sometimes permissions this can break if something got installed as root, which might be the case for pycparser on your system.

Binary xPacks are plain archives, you download them and extract in any folder

Very nice, I didn't realise that!

And yes, pyocd really needs better documentation. The user base has grown enormously over the past 2 years (especially since STLink support was added) and the need for better docs has been ever more apparent. In the background I'm working on a pyocd.io site and more docs.

[ilg-ul]

I'm working on a pyocd.io site

Why not a pyocd.github.io? It is free and very easy to manage.

[flit]

In fact, I'm planning to use GitHub Pages to serve the site (built with Jekyll), but I'd prefer a custom domain. (Until recently pyocd didn't have its own organisation as it was mixed in with Mbed OS projects, so it couldn't have a 2nd-level domain with GH Pages, though this wouldn't be an issue any more.)

[ilg-ul]

anyway, if you host it at github, you can take a look how the Embed CDT web is automatically generated by some GitHub Action jobs.

plus that there is a web-preview site, which is automatically generated when I push changes to the develop branch.

the xPack and the µOS++ sites too, I recently updated them all (previously I used Travis, but it became very slow).

once configured, everything is automatic, I work on the develop branch, check the results, and when ready, I merge develop to master.

very convenient.

regards,

Liviu

[flit]

Definitely will look, thanks for the pointers.

[jonahgraham]

(This bug has gone on quite a few tangents - this comment is back the the original title.)

I don't know how used the standalone debugger is. There have been discussions about discontinuing it from CDT, but no action on actually removing it. There have been less than 400 downloads of the CDT 10.0 version since it was released in Sep. As a comparison, the C/C++ package has >850,000 downloads. So no one is really investing in this area at the moment. However the C/C++ package can be used as the standalone debugger, and I am not sure how many people use it that way.

However, the important feature the standalone debugger brought to CDT was "project-less" debugging. That was a critical feature for CDT and just requires a user to be able to create and use a launch configuration without an associated project.

The standalone debugger creates a workspace automatically without prompting the user, but does not create projects. To navigate the source files it uses the Executables view which parses the elf (or similar) file for debug info to extract the sources that make up the executable. I added a Debug Sources view in 2019-12 that does the same kind of thing, but uses gdb to list the sources being debugged, which is more reliable and does not require CDT to parse elf files. See https://wiki.eclipse.org/CDT/User/NewIn910#Debug for more info on the latter.

The launch configurations (I tried QEMU one) seem to work fine without a project - just that the defaults that rely on project settings need to be set in the launch configuration instead.

My conclusion is that there isn't anything worthwhile doing on this issue, assuming the project-less debugging is indeed working. If it is not working, I think specific bugs on the failing paths would be useful.

Therefore, I recommend closing this for now.

[flit]

@jonahgraham Thanks for all the detailed information and history. It's good to know the standalone debugger is likely to be removed.

One question I have is what the difference is between CDT and the C/C++ package? Does the former mean installing CDT via Help > Install New Software…, and the latter the "Eclipse IDE for C/C++ Developers" in Eclipse Installer?

I do think the embedded space is potentially different from native C/C++ debugging. Imo, there is a place for a stripped down Embed CDT focused on debugging. Most of the embedded developers I know dislike Eclipse primarily because of all the baggage included with a standard install.

However, I'm fine with closing this issue. Sounds like a standalone Embed CDT might better be something new rather than based on the CDT standalone debugger.

I'll definitely take a look at the Debug Sources view, and will test and work on any issues with project-less debugging.

[ilg-ul]

what the difference is between CDT and the C/C++ package?

Currently the Embedded CDT package uses the same set of plug-ins as CDT, plus the Embedded CDT plug-ins.

[jonahgraham]

what the difference is between CDT and the C/C++ package

CDT is the Eclipse project (collection of plug-ins) that support C/C++ development.

The C/C++ Package is the product that contains many Eclipse plug-ins beyond just CDT that are C/C++ focused and packages them as a single download for the user. The main ones being CDT, but also include TraceCompass, Linuxtools and others. See product release info (expand Detailed features list for all the features in the C/C++ Package)

As Liviu said above, the Embedded C/C++ Package is mostly the same as the C/C++ Package, but targets embedded engineers by including the Embedded CDT project's plug-ins too, along with other customizations that Liviu's experience over the years has helped shape a product that is more embedded focused.

[flit]

Thanks for the clear explanation, Jonah.

[jonahgraham]

I'll definitely take a look at the Debug Sources view, and will test and work on any issues with project-less debugging.

@flit please have a look at #483 - it looks like there are at least some issues with project-less debugging.