Frequent crashes when debugging

[davetansley]

Describe the bug While debugging with ZESaruX v0.8 (stable) I get frequent crashes, where ZESaruX will terminate and the debugging will stop. There doesn't seem to be any commonality with these crashes, they seem to happen randomly. Sometimes while stepping through instructions, sometimes while paused on a single instruction. They are not corrolated with crashes of the Spectrum code.

To Reproduce Steps to reproduce the behavior:

1. Debug code as normal

Expected behavior Stable debugging.

Version etc. (please complete the following information):

• SW Version: 0.9.1
• OS: Windows 10
• ZEsarUX SW Version: 0.8

Additional context { // Verwendet IntelliSense zum Ermitteln möglicher Attribute. // Zeigen Sie auf vorhandene Attribute, um die zugehörigen Beschreibungen anzuzeigen. // Weitere Informationen finden Sie unter https://go.microsoft.com/fwlink/?linkid=830387 "version": "0.2.0", "configurations": [ { "type": "z80-debug", "request": "launch", "name": "ThePit", "zhostname": "localhost", "zport": 10000, "listFiles": [ { "path": "memory.list", "useFiles": true, "asm": "sjasmplus", "mainFile": "main.asm" } ], "startAutomatically": false, "skipInterrupt": true, "commandsAfterLaunch": [ //"-sprites", //"-patterns" ], "disassemblerArgs": { "esxdosRst": true }, "rootFolder": "${workspaceFolder}", "topOfStack": "stack_top", "load": "ThePit.sna", "smallValuesMaximum": 513, "tmpDir": ".tmp" }

]

}

[maziac]

Could you please try with the latest zesarux beta here: https://github.com/chernandezba/zesarux/releases/tag/8.1-16122019 AFAIR the stability of the socket interface had been improved.

[dgwyer]

I'm getting exactly the same behavior on Windows 10. Code compiles fines (I'm also using sjasmplus) but when I try to use the debugger I get very frequent, seemingly random, crashes.

I've tried using v8.0 stable, and the latest v8.1 beta and both exhibit this behavior.

[davetansley]

With both the latest dezog and the latest zesarux, I get the same (though the new features in the latest dezog are great!)

One other thing I noticed was that the register view seemed to mix up registers (especially b and a) in the top left panel. I've tried to repro this with a simple programme, but haven't had much luck. Will continune to try.

(Should also add that, even with the crashing, this tool is just utterly indispensible - couldn't even imagine how much time it has saved me debugging issues! Great work!)

[maziac]

Nice to hear that someone is using the tool (and that you like it :-) ). Sorry that you encounter those problems.

With the registers: there was once a bug where I parsed the wrong register in the response from Zesarux. But that should be fixed now. There is still the possibility that the registers are wrong especially if you change from reverse debugging to normal debugging or vice versa. Please inform me once you have found a reproducible way. DeZog is still in the testing phase regarding the reverse debugging. Please let me know if you find a problem there.

Regarding the crashes: My development environment is macOS. For me it works rather stable. I see Zesarux crashes only now and then. It normally happens at startup of debugging when zesarux still has a window open. With windows open zesarux might respond slowly and eventually terminates. Maybe this is something to look for when it happens again.

BTW if you haven't a window opened by yourself this might be done by zesarux itself when you are using memory breakpoints (WPMEM, not he normal PC breakpoints) and the connection to zesarux has been lost for some reason.

[dgwyer]

Thanks for the feedback. It seems to be consistently crashing even if I start ZEsarUX and then reset the emulator machine immediately prior to a new debugging session.

I can confirm it seems to be more stable on macOS. I got the setup info from this page:

http://www.breakintoprogram.co.uk/programming/assembly-language/z80/z80-development-toolchain

Dean uses it on macOS and is fairly stable for him. It's not an option for me to use macOS so I'm wondering what else I can try? Or do I wait until you can perhaps look into this further? Thanks for any help. I really like this debugger inside VS Code and don't really want to have to look for an alternative.

[maziac]

It's important that zesarux does not show any open window when you start the debug session (just in case...)

If you have it consistently crashing could you maybe attach your project (your asm sources) here together with the files in the .vscode folder.

(And thanks for the link, haven't seen the post before)

[dgwyer]

Prior to a debug session I've tried restarting the emulator machine via Machine->Sinclair Reasearch->ZX Spectrum 48K, and making sure no menus are open, and the footer is not visible. Still crashes.

As an example, here is a very simple code file I just tested:

DEVICE ZXSPECTRUM48

Code_Start:     EQU 0x8000
                ORG Code_Start

        ld hl, 16384
        ld (hl), 255
        ret

                SAVESNA "test1.sna", Code_Start

Start debug session, and it crashes instantly. I can assemble the code files using sjasmplus and load them in manually into the emulator. I've not had a single issue doing this so there's something goign wrong when a debug session is created.

Here are the launch and task VS Code files I'm using.

launch.json

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
            {
                    "type": "z80-debug",
                    "request": "launch",
                    "name": "Z80 Debugger",
                    "zhostname": "localhost",
                    "zport": 10000,
//          "topOfStack": "stackend",
                    "rootFolder": "${fileDirname}",
                    "listFiles": [
                        {
                                "path": "${fileDirname}/${fileBasenameNoExtension}.lst",
                                "useFiles": true,
                                "asm": "sjasmplus",
                                "mainFile": "${fileDirname}/${fileBasenameNoExtension}.z80"
                        }
                    ],
                    "disassemblerArgs": {
                            "esxdosRst": true
                    },
                    "load": "${fileDirname}/${fileBasenameNoExtension}.sna",
                    "skipInterrupt": true,
                    "startAutomatically": false,
                    "preLaunchTask": "sjasmplus"
            }
    ]
}

tasks.json

{
    // See https://go.microsoft.com/fwlink/?LinkId=733558
    // for the documentation about the tasks.json format
    "version": "2.0.0",
    "tasks": [
        {
            "label": "sjasmplus",
            "type": "shell",
            "command": "clear;sjasmplus",
            "args": [
                "--lst=${fileDirname}/${fileBasenameNoExtension}.lst",
                "${file}"
            ],
            "group": {
                "kind": "build",
                "isDefault": true
            }
        }
    ]
}

[maziac]

Thanks, will try this. BTW here is a tutorial from Cesar Wagener. For him the Windows setup seemed to work. https://github.com/maziac/DeZog/blob/master/documentation/extras/Tutorial_ZEsarUX_sjasmplus_z80-debug/IDE_Spectrum_English.pdf But the difference might be that he compiled zesarux on his own.

[dgwyer]

Thanks for the link. I won't have time to go through it until next week. In the meantime, please let me know if you make any progress for the Windows version. Would be great to have this more stable. :)

[maziac]

I could reproduce the zesarux crash on Windows 10. The same version is running fine under macOS. I informed Cesar, but I guess it will take some time until he can look into it as he is busy finishing the 8.1 version. BTW also the sample program from C. Wagener is crashing zesarux for me. For him it obviously worked on Win 10. So maybe it's worth a try to compile zesarux manually on Windows as outlined in his tutorial (link above).

[davetansley]

Just an update on this - I updated to the new 8.1 release of zesarux yesterday, updated the debugging extension, and debugging seems stable now. No more crashes on startup. Woohoo!

[dgwyer]

Thanks for the update Dave. I'll test my end and post feedback here too.

[dgwyer]

I've just tested a very simple program and it's still crashing every single time I try to run the debugger, unfortunately with the new v8.1 release.

[davetansley]

I wonder what I'm doing different... are you using the latest version of the extension? (the "dezog" one rather than the "z80-debug" one?)

[dgwyer]

Hmm, OK let me compare with your setup.

In VS Code I have the following extensions installed:

• Z80 Assembly v0.0.3 by Imanolea
• Z80 Debugger v0.9.1 by maziac

Here is my launch.json:

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "type": "z80-debug",
            "request": "launch",
            "name": "Z80 Debugger",
            "zhostname": "localhost",
            "zport": 12000,
//          "topOfStack": "stackend",
            "rootFolder": "${fileDirname}",
            "listFiles": [
                {
                        "path": "${fileDirname}/${fileBasenameNoExtension}.lst",
                        "useFiles": true,
                        "asm": "sjasmplus",
                        "mainFile": "${fileDirname}/${fileBasenameNoExtension}.z80"
                }
            ],
            "disassemblerArgs": {
                    "esxdosRst": true
            },
            "load": "${fileDirname}/${fileBasenameNoExtension}.sna",
            "skipInterrupt": true,
            "startAutomatically": false,
            "preLaunchTask": "sjasmplus"
        }
    ]
}

And my tasks.json:

{
    // See https://go.microsoft.com/fwlink/?LinkId=733558
    // for the documentation about the tasks.json format
    "version": "2.0.0",
    "tasks": [
        {
            "label": "sjasmplus",
            "type": "shell",
            "command": "clear;sjasmplus",
            "args": [
                "--lst=${fileDirname}/${fileBasenameNoExtension}.lst",
                "${file}"
            ],
            "group": {
                "kind": "build",
                "isDefault": true
            }
        }
    ]
}

Can you spot any obvious differences?

[maziac]

An update from me: I tested the following combination on Windows 10:

• vscode 1.42.1
• DeZog 0.12.2
• Zesarux 8.1
• The z80-sample-program

I get an immediate crash ("kernel panic") of zesarux.

But for me I can mitigate this behavior a little bit: Different to what I said before I click once into zesarux to show a window (the first one which says "F10 Exit Emulator" at the bottom). Now while the window is being displayed I start the debug session from vscode. And (in most of the cases) zesarux does not crash and the debug session starts normally. (Maybe the window slightly changes the timing. Furthermore I had disabled the zesarux tooltips Settings->GUI->Tooltips, just in case.)

Could you maybe check if that works for you as well.

[maziac]

I did some more tests. Could you try out the below special version of 0.12.2 (unzip and install from vscode). dezog-0.12.2_zrcp_delay_100ms.vsix in https://github.com/maziac/DeZog/releases/edit/0.12.2

I added a pause between all ZRCP commands of 100ms. This seem to work somewhat reliably for me (although of course the whole debugger becomes quite slow). So, it's not a final solution but I wanted to see if this helps in your case as well.

I tried smaller pauses (10ms, 1ms) but these don't work reliably.

Alternatively you could also try out the new internal z80 simulator in 0.12.2: "remoteType": "zxsim".

[dgwyer]

I tried the debugger you suggested and while it doesn't crash, I can't get it to work. It doesn't seem to run the program. I just see the grey screen.

Here's a video (GIF) that shows my launch.json and output.

[maziac]

I will come back to that zxsim issue, but what about Zesarux with dezog-0.12.2_zrcp_delay_100ms.vsix ?

[davetansley]

Have to say, I'm a little confused about what I'm doing right... I've had consistently good debugging since I upgraded zesarux the other day. I've seen maybe 1 or 2 stripey screen crashes in many hundreds of restarts. I'm not doing anything particularly different - sometimes zesarux is visible, sometimes behind VS. The reverse step functionality works GREAT!

So here's my set up:

zesarux 8.1 ASM Code Lens 1.4.1 DeZog 0.11.0 (from VSIX) Z80 Assembly 0.0.3 Z80 Debugger 0.9.1 (disabled)

My launch.json:

{ // Verwendet IntelliSense zum Ermitteln möglicher Attribute. // Zeigen Sie auf vorhandene Attribute, um die zugehörigen Beschreibungen anzuzeigen. // Weitere Informationen finden Sie unter https://go.microsoft.com/fwlink/?linkid=830387 "version": "0.2.0", "configurations": [ { "type": "dezog", "request": "launch", "name": "ThePit", "zhostname": "localhost", "zport": 10000, "listFiles": [ { "path": "memory.list", "useFiles": true, "asm": "sjasmplus", "mainFile": "main.asm" } ], "startAutomatically": true, "skipInterrupt": true, "commandsAfterLaunch": [ //"-sprites", //"-patterns" ], "history": { "reverseDebugInstructionCount": 10000, "codeCoverageEnabled": false }, "disassemblerArgs": { "esxdosRst": true }, "rootFolder": "${workspaceFolder}", "topOfStack": "stack_top", "load": "ThePit.sna", "smallValuesMaximum": 513, "tmpDir": ".tmp" }

]

}

My tasks.json:

{ "version": "2.0.0", "tasks": [ { "label": "sjasmplus", "type": "shell", "command": "sjasmplus", "args": [ "--lst=memory.list", "--fullpath", "main.asm" ], "problemMatcher": { "fileLocation": [ "relative", "${workspaceRoot}" ], "pattern": { "regexp": "^(.)\((\d+)\):\s+(warning|error):\s+(.)$", "file": 1, "line": 2, "severity": 3, "message": 4 } }, "group": { "kind": "build", "isDefault": true } } ] }

[maziac]

@dgwyer: Could you please test the new 0.12.3: https://github.com/maziac/DeZog/releases/tag/0.12.3

I tested your program, it works for me. But maybe the problem is that you immediately start the program.

Try to use "startAutomatically": false, in your launch.json and step over your code.

Otherwise as soon as your program reaches the RET instructions it will go somewhere and maybe distort the expected result.

[maziac]

For all who have problems with zesarux/DeZog on Windows could you please try if this works for you.

I had a reproducible crash on every start of a debug session. Then I started zesarux from powershell with argument "--verbose 3" to see if there is any helpful output. But with this change the crash does not happen anymore for me. I then tried other verbosity level and with all other levels no crash.

But now the distorting thing: Now even when I used a verbosity of 0 there is no crash anymore. Alsoalso no crash when I start zesarux directly (without powershell).

I don't get it to crash anymore. Maybe zesarux is storing some state information somewhere?

But anyhow please try and let me know if this helps for you as well or if is only working for myself.

[dgwyer]

Is there a special way to run zesarux from the command line?

When in the same dir as the executable I enter zesarux.exe --verbose 3 but it won't run.

[maziac]

Yes, should be as simple as this_

zesarus.exe --verbose 3

called in same directory as zesarux from PowerShell (or cmd). What does it mean it won't work. Is the exe not found? Alternatively you could also change the verbosity in the GUI of zesarux, when you start zesarux as you normally do.

[dgwyer]

I just checked via the GUI and verbose was already set to 3.

[maziac]

Could you change it to e.g. 4 and try again.

[kborowinski]

@maziac I've tested verbose levels 0 to 4 and it did not help. I get kernel panic every single time when debugging with zesarux on Windows since version 8.1. It's so frustrating that I've ditched zesarux in favor of CSpect emulator because it's rock solid in comparison to zesarux. The pitty of course is that CSpect does not support any remote debugging but to be honest at this point I can't remote debug with zesarux on Windows anyways.

BTW I have noticed that since DeZog 0.13.2 the debugging with zsim is 2 times slower than on 0.13.1. What gives?

[maziac]

Yes, zsim got slower. That's mainly because I added watchpoint handling and history. But zsim is anyway not build for speed.

For CSpect: my plan is to support also CSpect in the future. The problem at the moment is that I don't get the UART plugin to work. For some people the plugins work for others not. Unfortunately it does not work for me. I hope that it may work in the next CSpect release (hope that something is released when he is ready with wolfenstein).

For Zesarux: This was just a small hope. Because for me it is now working. I had a reproducible crash before every time I started a debug session but it's gone now completely even if I change the verbosity level back to 0.

Maybe zesarux is saving some configuration sometimes. Or it has something to do that I started it first from powershell. All very strange...

[kborowinski]

@maziac I don't know if that helps but I found that zesarux does not panic so often when I run in on i7 8700K (system I have at work) than on my home system based on i7 990x (both are running latest Windows 10 versions). Some guys on discord are reporting that zesarux is much more stable when running on linux.

[maziac]

@kborowinski Could you please try the 0.13.3 and what is mentioned there: https://github.com/maziac/DeZog/releases/tag/v0.13.3

(Unfortunately I cannot test myself as Zesarux is working for me.)

[kborowinski]

@maziac I have just installed 0.13.3 and followed all the steps of smart load test and Zesarux does not crash anymore!

[maziac]

OK, cool! So it seems to have something to do with the loading. Although I still don't know what this is. Maybe you could try for a few days to see that the crash does not re-appear.

[kborowinski]

@maziac Question? Does 0.13.3 support loading files in nex format? I can load and debug sna files but when loading ZX Spectrum Next specific nex format I get errors from DeZog regarding extended-stack and cpu-code-coverage not being enabled? Also if you look at the debug console you can see that PC is set to 11DC instead of proper location. Is it to be expected in test version?

[maziac]

In fact it shouldn't make any difference to DeZog. The command bypasses DeZog and goes straight to zesarux. But I must admit that I have only tested a sna file. I will test a nex File tomorrow.

[maziac]

The PC should be right after one step. Don't know what is happening there.

At least it is not crashing :-)

[kborowinski]

@maziac Yep, no kernel panics so far ;)

OK, the PC is right after few steps however DeZog displays following error on each step and then connection to Zesarux dies eventually (it happens only when smart loading nex file):

I'll do more tests with sna and nex.

[maziac]

Hi,

it seems that loading a nex file resets some of the former set variables. So this is a side-effect.

I have here a new release (0.13.4, https://github.com/maziac/DeZog/releases/tag/v0.13.4) so you can experiment a little bit. DeZog now loads the sna/nex file itself (as before 0.13.3) but now you get 2 parameters to set a wait time before and after loading.

Maybe you can experiment a little bit what works for you. I would start with 2000 (ms) for both values to see if it works at all. Then you can lower the one or the other.

Overall start time will of course increase by that time as well, so you can see that it is working.

In your launch.json you have to add:

    "load": "your_nex_file.nex",

    "debug_wait_before": 2000,
    "debug_wait_after": 2000,

BTW zsim should have become a little faster in this release.

[kborowinski]

@maziac:

1. I did several trials and found than on my system minimum safe delay for debug_wait_before is 750 ms - anything less is causing Zesaurx kernel panics. I didn't need to specify debug_wait_after. Great job mate! Any suspicions why you need to introduce delay? Is it delay before you send data to Zesarux?

2. As for zsim it does not work for me in this release. To be precise, zsim starts, I can see the memory and screen panel but debugging does not work at all.

Edit:

And also VSC is complaining that debug_wait_before is not allowed - but I guess that this is expected

[maziac]

That's great news. At least some workaround exists now. I will leave the configuration option in for some time. It's normal that vscode complaints as it is not available "officially" (package.json).

The debug_wait_before is a delay before DeZog sends the "smartload" command to zesarux. Before and after it is sending other commands, e.g. breakpoints, code coverage, call stack, registers etc. So maybe we need to give zesarux a little time to settle down before loading a program. But still it's strange that this is only on windows.

I have seen that the current wait time is not so precise (compared to the previous command). So I generated another release 0.13.5 (https://github.com/maziac/DeZog/releases/tag/v0.13.5). Maybe you could test the timings there as well.

For zsim: Maybe something was broken in the 0.13.4, sorry. I'm currently doing a lot of changes in that area that also affect zsim. Please try again in 0.13.5.

I really appreciate your testing efforts. It is a great help for me. 👍

[kborowinski]

@maziac No problem mate. I'm glad I could help. In fact thank you for all the work, I use your extensions on daily basis. BTW I've tried latest build of DeZog (I build directly from master repository) and zsim still does not work

[maziac]

That is strange. zsim is working for me. Seems that either zesarux or zsim is working for you, but never both. Do you maybe have a short project for me with a launch.json file?

[kborowinski]

@maziac With the 0.13.5 I was able to go down to 50 ms with debug_wait_before parameter before Zesarux started kernel panics again.

Here you have the code and launch.json zxnext.zip

Edit: You are probably right regarding the Zesarux boot delay. As you said, maybe it's worth leaving that parameter in launch.json for some time because we don't know how much of the delay other Windows boxes may require.

[maziac]

zsim: I found the problem. An unhandled exception. I will do some more tests and do a new release.

zesarux: I got a fix from Cesar. Maybe you could give it a try. Remove the "debug_wait_before" or set it to 0 and use zesarux_win.zip

Here are the install instructions from Cesar: "Download the last windows stable build 8.1; then uncompress the [zip] file, rename zesarux.ex_ to zesarux.exe and copy it on the stable build 8.1 (overwrite the exe). There are some dll but you should also have it on the stable 8.1"

[kborowinski]

@maziac I saw that Cesar made some fixes to zrcp protocol in last few commits. If that's the fix that Cesar is talking about then I should have already updated version of Zesarux. I do compile new version every time Cesar pushes new commit.

Edit: Nope, the fix you got does not work. I get kernel panics without debug_wait_before parameter, so do not remove that option from launch.json for foreseeable future, please.

[maziac]

Yes, I think it's the same patch.

And, no worries, I'll leave the workaround in. Will rename it to a proper parameter and remove the debug_wait_after.

[kborowinski]

@maziac Does zrcp support screen refresh? When you debug with Zesarux the emulator screen is black or does not show complete content. Is there a way to force Zesarux to periodically update emulator screen while debugging?

[maziac]

I know the problem. For debugging it would be nice to see the screen always. ZEsarUX is emulating too exact. I.e. it displays only what has really been passed by the ray (virtual cathode ray).

I think the best you can get is to use "--show-electron-debug". This at least visualize the ray, so you know why you can't see anything (because you can see that the ray hasn't been there yet).

But maybe the best is to contact Cesar directly.

BTW the internal simulator always shows what is on the screen as it emulates no cathode ray :-)

[kborowinski]

@maziac I'll try the trick with scanline, maybe I can live with that for a moment. BTW do you plan to support 128K model in zsim?

[maziac]

Yes. Actually you can load a 128k sna file already. What's missing is the bank switching, but that's easy. What I don't plan is any audio support.

[kborowinski]

@maziac But it still loads only 48K ROM, does it?

Edit: BTW I just compiled 0.13.6 and zsim works again as well as zrcp.loadDelay ;)