search

Marus / cortex-debug

Visual Studio Code extension for enhancing debug capabilities for Cortex-M Microcontrollers
MIT License
1.01k stars 240 forks source link

pyOCD support probably not up do date #322

Closed tdjastrzebski closed 2 years ago

tdjastrzebski commented 4 years ago

I suggest improving pyOCD support. Since November 2018 and this commit pyocd-gdbserver as a startup command is deprecated. 0000782:WARNING:gdb_server:pyocd-gdbserver is deprecated; please use the new combined pyocd tool.

In the log attached below I see another problem Target type 'stm32f767zitx' not recognized. My Nucleo-767ZI board is auto-detected as target stm32f767zitx, which is unsupported. stm32f767zi target is supported but I do not find a way to override target type.

0000775:WARNING:gdb_server:pyocd-gdbserver is deprecated; please use the new combined pyocd tool.
0001007:ERROR:gdb_server:uncaught exception: Target type 'stm32f767zitx' not recognized. Use 'pyocd list --targets' to see currently available target types. See <https://github.com/mbedmicro/pyOCD/blob/master/docs/target_support.md> for how to install additional target support.
Traceback (most recent call last):
  File "c:\program files\python36\lib\site-packages\pyocd\board\board.py", line 61, in __init__
    self.target = TARGET[self._target_type](session)
KeyError: 'stm32f767zitx'
{
    "name": "Debug (pyOCD)",
    "type": "cortex-debug",
    "request": "launch",
    "servertype":"pyocd",
    "cwd": "${workspaceRoot}",
    "executable": "${workspaceRoot}/BUILD/NUCLEO_F767ZI/GCC_ARM-DEBUG/${workspaceRootFolderName}.elf",
    "device": "STM32F767ZI",
    "v1": false,
    "svdFile": "${workspaceRoot}/STM32F7x7.svd",
    "runToMain": true
}
haneefdm commented 4 years ago

I know. I already made the changes but did not push them out. Reason: it will be a breaking change for people who set a pathname for the server executable, and I don't have the bandwidth to deal with that.

So, I told myself, let's just stay in the compatibility mode until pyocd truly stops supporting it. Maybe I am wrong. When I look at pyocd source code, yes there are a lot of differences between the two different launch types

For now, you can create a shell script for pyocd-gdbserver to launch pyocd in the new way

Maybe the time has come to switch. I don't use pyocd on a regular basis but any advice is appreciated on what the usage model is. Do people specify a full pathname for us to find or just use the system path? I am talking Win/Linux/Mac

I like pyocd BTW, but they should not have deprecated that way.

tdjastrzebski commented 4 years ago

@haneefdm I think the question is how many people still use almost two-years-old pyOCD release 0.13 and earlier and how much new pyOCD functionality, fixes and improvements is not exposed by cortex-debug due to self-imposed limitations.
Frankly, I do not know the answer but I think it may be time to consider moving forward.
I am relatively new to embedded programming but my experience so far is rather painful, probably due to lack of years of experience. Whatever it is worth, cortex-debug, despite its very limited documentation and examples is the tool which allowed me to move forward. The value of hiding complexity is not to be underestimated.

If compatibility with older pyOCD versions is a real concern, I suggest popping up VS Code warning window for some time if detected pyOCD version is too old, before the change is made.

haneefdm commented 4 years ago

I will wait for @Marus to comment for a few days. Give him a chance to reflect

patrislav1 commented 4 years ago

If compatibility with older pyOCD versions is a real concern, I suggest popping up VS Code warning window for some time if detected pyOCD version is too old, before the change is made.

Another way would be to provide pyocd-gdbserver or pyocd-legacy as yet another server type, for the legacy pyOCD, if there are users still depending on it. Then the pyocd server type could be allowed to move forward - people who update Cortex-Debug but don't update pyocd then just have to change the server type in their launch.json.

stephendpmurphy commented 3 years ago

I second the updating of support for pyocd to use the latest and recommended gdb server commands.

In regards to overriding the target, I seem to have a solution seen below by adding the serverArgs array. My limited amount of testing appears to show it working and allows me to set breakpoint and step through my source. Take note that the --target and stm32f413rg are their own params in the serverArgs array.

{
    "name": "Debug (pyOCD)",
    "type": "cortex-debug",
    "request": "launch",
    "servertype":"pyocd",
    "cwd": "${workspaceRoot}",
    "executable": "./build/my-firmware.elf",
    "device": "stm32f413rg",
    "runToMain": true,
    "v1": false,
    "serverArgs": [
        "--target",
        "stm32f413rg"
    ]
}
newbrain commented 3 years ago

I'll add my two cents: The only reason pyocd-gdbserver is still around might be this very extension (BTW: thanks @Marus, it's great!). The other deprecated tools will probably be removed from next version. See the last point in this pyOCD PR.

I second @tdjastrzebski idea of a warning - if one is using such an old pyOCD, it's really time to upgrade.

Marus commented 3 years ago

I do think we probably should head towards the new interface for pyocd - yeah - while it may cause a few people to have to update configurations can't hold on to the past forever.

Like @haneefdm thought I don't typically use pyocd - although it looks like it now works with STLink (and possibly J-Link) hardware - so hopefully I should be able to get this running to try out things and hopefully update

tdjastrzebski commented 3 years ago

I know this may be a bit provocative but several months have passed and, while I am still new to embedded programming, my perspective has changed a bit.
Why not to drop pyOCD support and continue with OpenOCD only? pyOCD seems to be good only for nucleo (but not all) and similar boards but not for custom hardware while OpenOCD is good for both. Drop stutil support as well. It's last release was almost a year ago, not much new development, no support for new MCUs like STM32H7A3. The only thing I would add to cortex-debug is flash support for OpenOCD. If license permits, I would consider redistributing cortex-debug with OpenOCD - that means fast start for new enthusiasts and reliable debugging = good experience. Is not it all what counts?

RisinT96 commented 3 years ago

What do you mean pyOCD support is only good for nucleo?

It supports any Cortex M MCU, while they might not have out-of-the box support for all MCUs, you can very easily add support for new ones with industry standard CMSIS packs.

On the contrary, I've had better experience with pyOCD than OpenOCD, OpenOCD is always such a pain to set up and launch.

newbrain commented 3 years ago

pyOCD seems to be good only for nucleo

[citation needed]

Really, I use cortex-debug + pyOCD with , let me check:

And this just from looking around on the bench...

newbrain commented 3 years ago

OpenOCD is always such a pain to set up and launch.

Yes, so true. I only use it as it comes precanned in VisualGDB. Every time I tried to set it up for myself I went mad.

tdjastrzebski commented 3 years ago

I meant: nucleo and similar. If I count it correctly, pyOCD supports only 14 ST processors, mostly those found in Nucleo boards. I think it is a bit surprising since, best to my knowledge, pyOCD development is lead by ST folks. I agree, OpenOCD is a bit pain, recent builds are difficult to find but, personally, I found it very stable and versatile.

patrislav1 commented 3 years ago

Here's a non-exhaustive list of devices covered by pyocd through cmsis packs. Good luck covering all those with openocd

https://www.keil.com/dd2/

haneefdm commented 3 years ago

My $0.02. pyOCD is actually pretty good and responsive to bug fixes. OpenOCD bugs leaks like crazy, not good at threaded apps., too many forks. A bug fix can take years to get resolved with OpenOCD

newbrain commented 3 years ago

If I count it correctly

Here is the rub, see @patrislav1 comment. I've yet to find something it would not handle. I never got OpenOCD working with the 43S67 for example (you are in a maze of twisty little passages configuration files, all alike)

pyOCD development is lead by ST folks.

As far as I know, it's tied to Arm, not ST. Its most active (currently) maintainer is, in fact, from Arm.

recent builds are difficult to find

And this (along with the atrocious complexity*) is really a problem. From their page (updated in 2018!):

OpenOCD is available in many different forms from numerous places, this page should serve as a guide directing you to the option most appropriate for the usecase.

(*) I don't deny it has (probably?) more features. I maintain they are difficult to use. pyOCD is extremely accessible.

tdjastrzebski commented 3 years ago

@newbrain Exactly, this is my point, OpenOCD has more features, supports more processors, seems to be much more stable - at least this is my experience. But the same time it is more complex and less accessible.
And here is where cortex-debug comes into play!

RisinT96 commented 3 years ago

How is it more stable and better supported if you have to read an entire wiki just to find which version you need to download? Are you just trolling us?

newbrain commented 3 years ago

I think you should reevaluate your statement about "supports more processors". Did you check the link in @patrislav1 comment? Is there any mainstream or even obscure Arm Cortex-M missing? Did you count them? The total is 7173 different devices.

Yes, pyOCD is for Arm Cortex-m. That, and the fact it does not really support JTAG (yet) are the only thing missing, IME.

Note that we are discussing this on an extensions called "cortex-debug" so I would expect the lack of support for other architectures to be the epitome of moot point!

tdjastrzebski commented 3 years ago

All right, that was just an idea, probably not the best one. What I am missing from pyOCD is support for STM32H7A3 and OpenOCD supports this chip without problems.

newbrain commented 3 years ago

support for STM32H7A3

Ok, at this point I totally agree with @RisinT96:

STM3H7A3

tdjastrzebski commented 3 years ago

Well, pyocd list --targets does not mention STM32H7A3. Where does this list come from?

newbrain commented 3 years ago

Now we are talking, and possibly, listening... I ask you (again) to check the link: https://www.keil.com/dd2/ All the MCUs in that list are supported by pyocd, you just need to issue: > pyocd pack -i <<desired model>> (see below for more complete help)

This is the result:

(venv) ~\Git\pyOCD [LinuxLibUSB ≡]> pyocd pack -i stm32h7a3
Downloading packs (press Control-C to cancel):
    Keil::STM32H7xx_DFP::2.7.0
(venv) ~\Git\pyOCD [LinuxLibUSB ≡]> pyocd list --target | Select-String 'stm32h7a'    
  stm32h7a3agixq            STMicroelectronics      STM32H7A3AGIxQ               STM32H7 Series, STM32H7A3   pack     
  stm32h7a3aiixq            STMicroelectronics      STM32H7A3AIIxQ               STM32H7 Series, STM32H7A3   pack
  stm32h7a3igkx             STMicroelectronics      STM32H7A3IGKx                STM32H7 Series, STM32H7A3   pack
  stm32h7a3igkxq            STMicroelectronics      STM32H7A3IGKxQ               STM32H7 Series, STM32H7A3   pack
  stm32h7a3igtx             STMicroelectronics      STM32H7A3IGTx                STM32H7 Series, STM32H7A3   pack
  stm32h7a3igtxq            STMicroelectronics      STM32H7A3IGTxQ               STM32H7 Series, STM32H7A3   pack
  stm32h7a3iikx             STMicroelectronics      STM32H7A3IIKx                STM32H7 Series, STM32H7A3   pack
  stm32h7a3iikxq            STMicroelectronics      STM32H7A3IIKxQ               STM32H7 Series, STM32H7A3   pack
  stm32h7a3iitx             STMicroelectronics      STM32H7A3IITx                STM32H7 Series, STM32H7A3   pack
  stm32h7a3iitxq            STMicroelectronics      STM32H7A3IITxQ               STM32H7 Series, STM32H7A3   pack
  stm32h7a3lghxq            STMicroelectronics      STM32H7A3LGHxQ               STM32H7 Series, STM32H7A3   pack
  stm32h7a3lihxq            STMicroelectronics      STM32H7A3LIHxQ               STM32H7 Series, STM32H7A3   pack
  stm32h7a3nghx             STMicroelectronics      STM32H7A3NGHx                STM32H7 Series, STM32H7A3   pack
  stm32h7a3nihx             STMicroelectronics      STM32H7A3NIHx                STM32H7 Series, STM32H7A3   pack
  stm32h7a3qiyxq            STMicroelectronics      STM32H7A3QIYxQ               STM32H7 Series, STM32H7A3   pack
  stm32h7a3rgtx             STMicroelectronics      STM32H7A3RGTx                STM32H7 Series, STM32H7A3   pack
  stm32h7a3ritx             STMicroelectronics      STM32H7A3RITx                STM32H7 Series, STM32H7A3   pack
  stm32h7a3vghx             STMicroelectronics      STM32H7A3VGHx                STM32H7 Series, STM32H7A3   pack
  stm32h7a3vghxq            STMicroelectronics      STM32H7A3VGHxQ               STM32H7 Series, STM32H7A3   pack
  stm32h7a3vgtx             STMicroelectronics      STM32H7A3VGTx                STM32H7 Series, STM32H7A3   pack
  stm32h7a3vgtxq            STMicroelectronics      STM32H7A3VGTxQ               STM32H7 Series, STM32H7A3   pack
  stm32h7a3vihx             STMicroelectronics      STM32H7A3VIHx                STM32H7 Series, STM32H7A3   pack
  stm32h7a3vihxq            STMicroelectronics      STM32H7A3VIHxQ               STM32H7 Series, STM32H7A3   pack
  stm32h7a3vitx             STMicroelectronics      STM32H7A3VITx                STM32H7 Series, STM32H7A3   pack
  stm32h7a3vitxq            STMicroelectronics      STM32H7A3VITxQ               STM32H7 Series, STM32H7A3   pack
  stm32h7a3zgtx             STMicroelectronics      STM32H7A3ZGTx                STM32H7 Series, STM32H7A3   pack
  stm32h7a3zgtxq            STMicroelectronics      STM32H7A3ZGTxQ               STM32H7 Series, STM32H7A3   pack
  stm32h7a3zitx             STMicroelectronics      STM32H7A3ZITx                STM32H7 Series, STM32H7A3   pack
  stm32h7a3zitxq            STMicroelectronics      STM32H7A3ZITxQ               STM32H7 Series, STM32H7A3   pack

(venv) ~\Git\pyOCD [LinuxLibUSB ≡]>

This is the internal help for the pack command:

(venv) ~\Git\pyOCD [LinuxLibUSB ≡]> pyocd pack -h  
usage: pyocd pack [-h] [-v] [-q] [-c] [-u] [-s] [-f GLOB] [-i GLOB] [-n] [-H]

optional arguments:
  -h, --help            show this help message and exit
  -v, --verbose         More logging. Can be specified multiple times.
  -q, --quiet           Less logging. Can be specified multiple times.

pack operations:
  -c, --clean           Erase all stored pack information.
  -u, --update          Update the pack index.
  -s, --show            Show the list of installed packs.
  -f GLOB, --find GLOB  Report pack(s) in the index containing matching device part numbers.
  -i GLOB, --install GLOB
                        Download and install pack(s) containing matching device part numbers.

pack options:
  -n, --no-download     Just list the pack(s) that would be downloaded, don't actually download anything.
  -H, --no-header       Don't print a table header.
(venv) ~\Git\pyOCD [LinuxLibUSB ≡]>
haneefdm commented 3 years ago

To all: pyOCD is not ST specific. It is ARM Cortex-M specific - may support Cortex-A, I don't know, supports multiple cores very well, funded by ARM (I think). I lent a hand last year and the ARM people were very supportive. Yes, it is still evolving but lot fewer bugs, very well written, modular.

If you want your device supported, talk to your MCU vendor. It probably already is.

Also, people using Mbed OS may want to use pyOCD, which is how I got sucked into it. The last thing I support is dropping pyOCD support. We can re-evaluate using pyocd-gdbserver because the startup python code between old and new has diverged a bit. We have to be careful

haneefdm commented 3 years ago

@newbrain Thanks

OpenOCD: I had to deal with Tcl scripts (config files), their C code. They don't even follow the proper gdb-sever protocol. Wrong on so many levels, but I got a working version for my company. Still not satisfied because these are generic problems that all devices can benefit from. Buffer overflows, I got that PR through. But there are so many more!?

patrislav1 commented 3 years ago

@haneefdm Fully agree, maybe except for the 'we have to be careful' bit. Python is a fast-moving ecosystem, and I think it's unlikely that people run the latest and greatest cortex-debug version together with a pyocd version from three years ago. And - maybe I'm not getting the full picture here - but I think even those users can just pip3 install --upgrade pyocd and be fine, right? Also quite ironic that for OpenOCD, a bleeding edge release candidate is recommended while for PyOCD an API is used that's deprecated since 2 1/2 years...

patrislav1 commented 3 years ago

By the way, I made this little wrapper script that fixes cortex-debug's issues with the current pyocd version. The changes necessary for cortex-debug should be minimal - I would open a PR myself, if I had any Typescript skills.

haneefdm commented 3 years ago

@patrislav1 :-) being careful is my default mode. 1.5 years ago, things were different, now pretty much you have to upgrade pyOCD very frequently to get bug fixes and new device support

newbrain commented 3 years ago

if I had any Typescript skills.

I understand TS as well as cuneiform, but I sometimes (after updates) just modify the debugadapter.js file to remove -gdbserver from the pyocd name (not even needed if you specify your own path) and instead add gdbserver to the array of arguments. Unfortunately, the latter can't be done using "args" in the launch.json, as it would end up in the wrong place (pyocd commands must be the first argument).

I would expect similarly simple changes in the TS source.

So: [js gibberish]serverExecutable(){return this.args.serverpath?this.args.serverpath:"pyocd-gdbserver"}serverArguments(){const e=this.ports.gdbPort,t=this.ports.consolePort;let r=["--persist","--port",e.toString(),"--telnet-port",t.toString()][more js gibberish]

Becomes: serverExecutable(){return this.args.serverpath?this.args.serverpath:"pyocd"}serverArguments(){const e=this.ports.gdbPort,t=this.ports.consolePort;let r=["gdbserver","--persist","--port",e.toString(),"--telnet-port",t.toString()] Ugly hack? Why would you say that...

flit commented 3 years ago

(I'm the pyocd maintainer.)

Sorry I missed out on the conversation here, I was really busy with work the last week. I won't try to reply to everything (not that it would be possible 😄 ), but maybe can answer a few things.

Me

I am indeed an Arm employee, working in the Research group. Mostly, pyocd is developed in my spare time, with a small amount on Arm time.

Pyocd and Arm

Pyocd was originally created and owned by the Mbed OS team at Arm. I was involved from almost the beginning, long before I joined Arm. In mid-2020, pyocd ownership was transferred to me so that it can become a true community project, and any community members can potentially become committers/maintainers. That's why the GitHub org changed to 'pyocd'.

Raison d'être

The goal (mission statement if you want to call it that) is for pyocd to be an easy to use debug tool and gdbserver for Arm-based* devices. A major part of pyocd is the REPL interface with good support for CoreSight low level debug (e.g. chip bringup) and the Python API. A common use case aside from development is in CI and other automated uses.

Device support

Currently pyocd only supports Cortex-M. There are plans to add AArch32 and eventually AArch64. All depends on available time. Help is welcomed! 😉

It's true that technically "any" Cortex-M device with a CMSIS-Pack is supported. However, there are, of course, issues with certain devices, and plenty of bugs. And the packs themselves are sometimes incorrect. If you have an issue, please report it! It's hard for me to keep up with every issue, but I do my best to address problems and fix bugs.

For reference, the target support docs show how to use CMSIS-Packs. I'm considering auto-install of packs, ie if you specify a part number for which a pack exists but is not installed (but unexpected, automatic network access is somewhat uncomfortable for me as a security researcher 😅 ).

Other

The state of pyocd docs is sorely recognized as lacking. 😢 I'm in the process of building a website (slowly), and then will start writing more docs. If anyone that wants to help, I would be massively appreciative!

I have a huge todo list and set of features I want to add. It will keep getting better.

If you have any other questions, probably should ask on via pyocd discussions.

haneefdm commented 3 years ago

@flit Thanks for chiming in Chris, and making things clearer

Chris, if I find something wrong, it is usually vendor code for programming, When I am able, I am willing to contribute. Especially in the core

jmc8 commented 3 years ago

Is anyone planning on driving this through to completion? It's been an open ticket for months, with a lot of talk, but I don't see any commits regarding the issue.

@haneefdm : You say you already made the changes but searching the repo yielded no results. Can you please tell me the commit rev/number your work is at?

Is there any way to simply override Cortex-Debug's hardcodes and call pyocd with my own command line?

haneefdm commented 3 years ago

See PR #435.

Is there any way to simply override Cortex-Debug's hardcodes and call pyocd with my own command line

Sure, you can write your own shell/bat script named pyocd-gdbserver that will intercept and do what you want. Note that the path is not hardcoded. Just the default name of the executable

Also, it does not solve the KeyError: 'stm32f767zitx' (original) problem because we have nothing to do with that. That is built into the pyocd install you have. We just pass the device name along. With pyocd, you don't even have to pass the device name. It can figure it out. If you need a device name, then ask your vendor what device name you are supposed to you. Or the pyocd repo

This has nothing to do with using the deprecated executable name.

haneefdm commented 3 years ago

To all, just upgraded the pyocd.