search

jdolinay / avr_debug

Source level debugger for Arduino - GDB stub for Atmega328 microcontroller used in Arduino Uno.
GNU Lesser General Public License v3.0
141 stars 33 forks source link

Interface between avr8-stub and platformIO #30

Open felias-fogg opened 3 years ago

felias-fogg commented 3 years ago

Hi Jan,

it seems that platformIO is the best IDE for integrating avr-stub (I just failed do to it in Sloeber/Eclipse). However, there are a number of funny things going on (you are not responsible for). Let me just enumerate the things I noticed:

  1. Global variables declared in the sketch are not visible in the debugger.
  2. Disassembly looks not convincing. There are no labels, names or line numbers. I am not sure whether they are right.
  3. While avr8-stub restricts flash breakpoints to 4, nothing like this happens in platformIO. So you can easily add more breakpoints but the debugger just never stops at the breakpoints with an index > 4. One probably has to make the breakpoints hardware breakpoints, because then there exists a command to restrict the number of such breakpoints. However, this would be sort of abusing terminology.
  4. When you use the continue command, then the debugger sends set-breakpoint commands for all breakpoints, and when execution stops, it sends remove-breakpoint commands for all breakpoints. This is really silly since it leads to wearing down the flash memory. I noticed it, when I implemented an error response when going beyond 4 breakpoints. When looking at the serial traffic, I was really surprised that such things are happening.
  5. When you reset and restart the program, it usually hangs at the first breakpoint. The reason is that when requesting reset, the debugger seems to set the first breakpoint. Only removing all breakpoints, using continue and then reset is a viable way to restart the program.

I guess, you are not the right person to report this to, but you my know, where I could post it.

Best, Bernhard

jdolinay commented 3 years ago

Hi Bernhard,

I agree that PIO is the best IDE for this though I still use eclipse for experiments, because I just feel I have more freedom in configuration - for example, I just change one workspace variable and I can try the avr-stub from different location.
If you want to try eclipse, in my experience it is still best to use the old AVR eclipse plugin and configure it as I described in this (old) article:https://www.codeproject.com/Articles/1003347/Creating-Arduino-programs-in-Eclipse. I found all the Sloeber plugins too limited in the options you can set.

To the points:

  1. Global variables declared in the sketch are not visible in the debugger. <

This is also true in eclipse so I thing it is feature of the GDB rather than some bug in PIO. Perhaps the avr-gdb does not properly respond to the requests from the IDE - they don't understand each other what global variable is. I find it a feature I can live with, not sure about PIO (VS Code) but eclipse remembers the expressions, so I only have to add the global variable once into the Expressions view and it will be there next time I open the project.

  1. Disassembly looks not convincing. There are no labels, names or line numbers. I am not sure whether they are right. <

I was actually surprised that PIO shows something, the disassembly never worked. It is (was?) a problem in the avr port of gdb. Look at the topic "Problem: The disassembly cannot be displayed" in the doc for the stub around page 75.

  1. While avr8-stub restricts flash breakpoints to 4, nothing like this happens in platformIO. So you can easily add more breakpoints but the debugger just never stops at the breakpoints with an index > 4. One probably has to make the breakpoints hardware breakpoints, because then there exists a command to restrict the number of such breakpoints. However, this would be sort of abusing terminology.

This is probably problem of the stub. There is some packet sent by gdb to evaluate the capabilities of the stub (qSupported I think) and perhaps it would need to implement response to this packet - to return the number of breakpoints supported. Another nice to have feature, I'd say :) It could be implemented, but it will cost memory and development time. The other option is that if BP is not working the user will look into the manual/code and find out there is this limit.

  1. When you use the continue command, then the debugger sends set-breakpoint commands for all breakpoints, and when execution stops, it sends remove-breakpoint commands for all breakpoints. This is really silly since it leads to wearing down the flash memory. I noticed it, when I implemented an error response when going beyond 4 breakpoints. When looking at the serial traffic, I was really surprised that such things are happening.

Yes, this is true. I believe this is feature of gdb. The point is that the user when examining the program in disassembly will see the original program, not the breakpoint instructions inserted instead of his instructions. There is optimization attempt in the stub to avoid rewriting the breakpoints - when gdb sends command to remove bps, the stub actually just notes the request but does not remove them. Then when gdb sets them again, the stub just sets those that are new or removes those that are not longer set, so the flash is not rewritten so much. Look at the gdb_insert_breakpoint and the macros like GDB_BREAK_ENABLE.

  1. When you reset and restart the program, it usually hangs at the first breakpoint. The reason is that when requesting reset, the debugger seems to set the first breakpoint. Only removing all breakpoints, using continue and then reset is a viable way to restart the program.

This may be something the PIO debugger could deal with. Perhaps there is some option for this or it depends on whether you request the debugger to load or attach to target - but it is probably hidden inside the PIO debugger. I think this may be default behavior of GDB and there is option for this in GDB - to insert breakpoint to main. Normally GDB thinks that you want to see the program stopped at the beginning of main when you start debugging. The support for PIO is actually created by @msquirogac so maybe he would know more about this.

Best, Jan

felias-fogg commented 3 years ago

Hi Jan,

I agree that PIO is the best IDE for this though I still use eclipse for experiments, because I just feel I have more freedom in configuration - for example, I just change one workspace variable and I can try the avr-stub from different location. If you want to try eclipse, in my experience it is still best to use the old AVR eclipse plugin and configure it as I described in this (old) article:https://www.codeproject.com/Articles/1003347/Creating-Arduino-programs-in-Eclipse. I found all the Sloeber plugins too limited in the options you can set.

I tried Eclipse and it took quite some time. One reason was that in the article it is not mentioned that you have to specify a baudrate argument in the call of avr-gdb. Then the version of avr-gdb I had on my Mac through homebrew was broken and nobody seems to care. And then it took quite a while to set all the right parameters. So, it is probably the most powerful IDE, but also one that takes quite some time when trying to learn about it. With PIO you are up to speed much faster.

To the points:

  1. Global variables declared in the sketch are not visible in the debugger. <

This is also true in eclipse so I thing it is feature of the GDB rather than some bug in PIO. Perhaps the avr-gdb does not properly respond to the requests from the IDE - they don't understand each other what global variable is. I find it a feature I can live with, not sure about PIO (VS Code) but eclipse remembers the expressions, so I only have to add the global variable once into the Expressions view and it will be there next time I open the project.

I guess, I have to live with it.

  1. Disassembly looks not convincing. There are no labels, names or line numbers. I am not sure whether they are right. <

I was actually surprised that PIO shows something, the disassembly never worked. It is (was?) a problem in the avr port of gdb. Look at the topic "Problem: The disassembly cannot be displayed" in the doc for the stub around page 75.

Actually, also the avr-gdb version of PIO for the Mac is somewhat broken in that the Python support does not work. With the fixed version I got, it works beautifully.

  1. While avr8-stub restricts flash breakpoints to 4, nothing like this happens in platformIO. So you can easily add more breakpoints but the debugger just never stops at the breakpoints with an index > 4. One probably has to make the breakpoints hardware breakpoints, because then there exists a command to restrict the number of such breakpoints. However, this would be sort of abusing terminology.

This is probably problem of the stub. There is some packet sent by gdb to evaluate the capabilities of the stub (qSupported I think) and perhaps it would need to implement response to this packet - to return the number of breakpoints supported. Another nice to have feature, I'd say :) It could be implemented, but it will cost memory and development time. The other option is that if BP is not working the user will look into the manual/code and find out there is this limit.

This turned out to be a longer story. So first, there is no way that you can tell gdb about an upper bound of allowed software breakpoints as far as I can tell from the docs. Second, I now implemented a check and stop execution immediately if there are too many breakpoints. Actually, because gdb might single-step over the current breakpoint, one might actually execute one instruction and end up somewhere (e.g., a function call might be actually executed). In addition, I use the debug_message function to send a warning. The tricky thing, however, is to implement the restriction in the right way - see below.

  1. When you use the continue command, then the debugger sends set-breakpoint commands for all breakpoints, and when execution stops, it sends remove-breakpoint commands for all breakpoints. This is really silly since it leads to wearing down the flash memory. I noticed it, when I implemented an error response when going beyond 4 breakpoints. When looking at the serial traffic, I was really surprised that such things are happening.

Yes, this is true. I believe this is feature of gdb. The point is that the user when examining the program in disassembly will see the original program, not the breakpoint instructions inserted instead of his instructions. There is optimization attempt in the stub to avoid rewriting the breakpoints - when gdb sends command to remove bps, the stub actually just notes the request but does not remove them. Then when gdb sets them again, the stub just sets those that are new or removes those that are not longer set, so the flash is not rewritten so much. Look at the gdb_insert_breakpoint and the macros like GDB_BREAK_ENABLE.

Ah! I did not look too closely, but now I understand. It is a clever way to reduce flash writing. But it also introduces the problem that you might not be able to set new breakpoints after deleting the old ones, although you are never going beyond the limit of 4 active breakpoints. Just try out the following: Set 4 different breakpoints and press continue. Now all breakpoints are written to flash, the execution will start, and after some time a breakpoint is reached. After that gdb sends the remove breakpoint commands and the stub deactivates the breakpoints. If you now delete all breakpoints in the IDE and set 4 new ones, these new breakpoints won't find there way into the array, because this is still filled with the old inactive breakpoints. And when starting execution again, you might wonder why the program does not stop at the breakpoints.

The solution is to make the array double as large so that it can keep track of the inactive and the new breakpoints. Add then add another entry so that we know that there were more breakpoints defined in the IDE than allowed. For the RAM breakpoints, things are, of course, simpler.

  1. When you reset and restart the program, it usually hangs at the first breakpoint. The reason is that when requesting reset, the debugger seems to set the first breakpoint. Only removing all breakpoints, using continue and then reset is a viable way to restart the program.

This may be something the PIO debugger could deal with. Perhaps there is some option for this or it depends on whether you request the debugger to load or attach to target - but it is probably hidden inside the PIO debugger. I think this may be default behavior of GDB and there is option for this in GDB - to insert breakpoint to main. Normally GDB thinks that you want to see the program stopped at the beginning of main when you start debugging. The support for PIO is actually created by @msquirogac so maybe he would know more about this.

When I saw that in the logic analyzer trace, I misinterpreted it. The debugger sends a set-breakpoint command after the reset, but this is never acknowledged by the stub (probably because the bootloader is still executing). The problem is caused by the stub because there might be still inactive breakpoints and the stub does not call gdb_update_breakpoints() before executing the reset. Probably just a simple omission since this it is done before executing kill and detach.

Finally, I was surprised that the value of an innocent looking local variable was not shown right. I assigned 100 to the integer variable i, but it showed as 36! I assigned 1000 to it and it showed as 808. As a global variable, as a local int8_t or int32_t variable, there was no problem. You probably know by now what the problem is, right? The local variable was stored on the top of the stack and your heuristic classified it as a potential return address where one has to mask some bits off. You called it an "ugly kludge" ;-). Disabling this part of the code helped. Now my question: Where did find the suggestion to mask out part of the word storing the return address? I went through some AVR MCU data sheets and the general Atmel AVR Instruction Manual and did not find any hint to it. I also have not heard about it before, but then in my age I do not hear everything anymore ;-).

Making a long story short: I have more fixes and changes to the manual. One should perhaps have a short section for Mac users ;-).

Cheers, Bernhard

jdolinay commented 3 years ago

Hi Bernhard,

Your updates are great! So I think the issues are actually solved.

To the 'ugly kludge', that is actually not my code and not my words, I had to look it up in the dictionary :) It comes from this project https://github.com/rouming/AVR-GDBServer. I don't remember if I verified the information at that time, probably I just took it for granted and I never noticed the problem with local variables. I was not able to find anything to confirm that information now either. So, as it works after your update with this code disabled, I guess it is not needed. Perhaps it applied to some older processors or they added the masking of the return address to avr-gdb code.

Thanks, Jan

felias-fogg commented 3 years ago

Hi Jan,

indeed very funny. In particular the phrase "they say you should mask out them, see Stack Pointer section at every AVR datasheet" is definitely not true of any of the current AVR MCU datasheets. Even in the supposedly first commercially available AVR MCUs, the AT90S8515, I did not found anything like it. So, I have no idea ... BTW, there is also masking code in the ISRs catching breaks and other debugger interrupts. But there it does not hurt, because it is clear that one deals with return addresses.

Best, Bernhard