Is there anyway to prevent the debugger from entering outside projects?

[miversen33]

To start, this is an amazing plugin :)

However I am noticing quite often that my breakpoints are treated more as suggestions as opposed to hard breakpoints. They aren't ignored, but the debugger likes to first stop inside the osv project before it stops in whatever code I have added a breakpoint to. Additionally, often the debugger will use the step over command to jump into some other running code before coming back to the section I care about (the next line in my project).

Its really cool to see all the different plugins that are running and such, but not really useful when I am trying to hunt down a bug in my own code. The Visual Studio debugger has an option for "justMyCode" where it doesn't traverse into "outside project" code, I was wondering if this is a thing I can enable for osv?

[jbyuki]

Thanks :) First I would like to mention that this is highly experimental and the implementation is far from pretty (lots of workarounds to make this work) so don't expect to run smoothly 100% of the time. It's more of a helpful tool to track down where the usual workflow is 1. place a breakpoint 2. inspect a runtime value 3. done. But if the basics are not even working, that's certainly an issue we can try to fix.

I never heard/experienced breakpoints stopping in osv itself but it's might be possible with an incorrect stackframe, in other words, the returned stackframe is going too deep into the osv code itself. I have never experienced it so it might be something configuration specific. I could also imagine that if you're not using the same luajit implementation, this could happen.

If that's the case, I don't think a "justMyCode" option would help. My theory is that osv is stopping at the right point in your project, but the stackframe is messed up and nvim-dap jumps you to the osv code. I remember that getting the correct level on the stackframe was me just trying out some numbers and finding something which seem acceptable but I never thought about it too much, nows the time maybe.. If you're interested the stackframe is filled here:

https://github.com/jbyuki/one-small-step-for-vimkind/blob/ccd671fedaca36e474aadfdd70b9ca4189fcd86e/src/handlers/stack_trace.lua.t#L26-L40

As you can see I added a (+2) for no real good reason. Might be a (+3) for your case. A deeper investigation into this might be good at some point.

Can you provide a log of a debugging session?

You could try to launch osv with logging. See :help osv on how to. The resulting log is located in nvim-data/osv.log (use echo stdpath('data') to locate nvim-data).

[miversen33]

Its hard to recreate as it is seemingly random. I have attached the requested log from a session in which the issue happened. It was the last thing that happened (in terms of OSV), but looking at the log, I can't make heads or tails of it so I don't know how to remove anything prior to that. osv.log

[jbyuki]

This is not enough to see where it's coming from. Can you try again with the nvim-dap logger and recreate the bug if it's possible? It is documented in the dap.txt documentation. It's missing some informations in the osv log because it's running multiple instances of neovim and the logger is not unified. But the nvim-dap logger gives the same thing actually and might be able to help us.

[jbyuki]

I have now implemented proper logging into osv as well, so you can try to log again with that... or with nvim-dap. However you like :)

[miversen33]

Sure no problem! I am noticing the issue tends to occur on a server reconnect. IE, I successful start a session, and then kill the connected neovim session. I restart it, restart the osv server and then reconnect my source code session to the osv server and it seems to jump into osv first (and the stack generally seems confused). I have attached the requested log, it is from a full successful usage followed by a session that jumped into the osv source instead of mine. osv.log

[jbyuki]

I haven't found the issue yet, but one thing that seem strange is that everything in the log is duplicated. I don't think that's normal. Like two clients are connected to the osv server, which it does not support by design. I'll investigate a bit more on it. Too bad I cannot reproduce the issue here but at least this log should help a bit.

[miversen33]

If you would like, you can reach out to me on discord miversen33#0131 and I would be more than happy to live walk through reproducing it for you/let you live troubleshoot it. If I get some time I might get a docker container thrown together to help demo the issue.

I want to reiterate, I really like this plugins, but this issue makes it a huge hassle for me to use. Thus if I can help eliminate said issue (its possible I am just doing something wrong, users are dumb afterall), it would greatly help out my work through my plugin development :)

[mtrajano]

Hello, I've actually been experiencing the same thing as @miversen33 . It seems to work correctly the first time I use the debbuger and then any following sessions keep running into the osv file (specifically this line) repeadetly which don't have breakpoints or other seeminly random parts of the codebase, for ex. lualine updating something. You mentioned this could be a luajit compatibility issue? Let me know if I could provide any more info, thank you for the great project, it's extremely helpful

[jbyuki]

Hello @mtrajano , yes we have run into the issue but couldn't reliably reproduce it and I cannot reproduce it at all using the config I use. More information on this issue would certainly help although I'm aware it's difficult. It could be another plugin interfering or something OS related. If you find something, let me know and I will try to provide a fix for it.

[mtrajano]

Hi @jbyuki, thank you for your response. Here is a screen recording of the issue happening and the logs when it happened. I'll keep looking to see if I can figure anything out but let me know if there's any other info you need from me and I would be happy to share.

https://github.com/jbyuki/one-small-step-for-vimkind/assets/4333381/fcc2fea2-883d-487b-b7f3-f8799458e43f

osv.log

NVIM v0.10.0
Build type: Release
LuaJIT 2.1.1716656478
Run ":verbose version" for more info

-- dap config
vim.keymap.set('n', '<leader>dv', function()
  require('osv').launch({ port = 8086, log = true })
end, { desc = 'Start debugging neovim' })
vim.keymap.set('n', '<leader>dV', function()
  require('osv').stop()
end, { desc = 'Stop debugging neovim' })

dap.configurations.lua = {
  {
    type = 'nlua',
    request = 'attach',
    name = 'Attach to running Neovim instance',
  },
}

dap.adapters.nlua = function(callback, config)
  callback({ type = 'server', host = config.host or '127.0.0.1', port = config.port or 8086 })
end

[mtrajano]

@jbyuki Looking at the logs it's possibly being caused by that offset you mentioned. It looks like on the first breakpoint it's hitting the correct line I set it's just stopping somewhere else further down in the stack frame. I'm not an expert on this by any means though.

The weird thing is is that it works sometimes. But more often than not it keeps stopping in that same line inside osv.

[jbyuki]

Hey @mtrajano , I have pushed a commit in an attempt to fix the issue. Let me know if it works, by closing this issue. Then this particular issue will be considered as fixed, otherwise other means will be needed to workaround it.

[mtrajano]

hi @jbyuki, looks like it's still breaking inside the osv file but this time in this line which doesn't show a path in the stack frame, here is the relevant part along with the entire log:

osv_new.log

{
  body = {
    stackFrames = { {
        column = 0,
        id = 1,
        line = 0,
        name = "main"
      }, {
        column = 0,
        id = 2,
        line = 0,
        name = "wait"
      }, {
        column = 0,
        id = 3,
        line = 1086,
        name = "Lua",
        source = {
          name = "@/Users/mtrajano/.local/share/nvim/lazy/one-small-step-for-vimkind/lua/osv/init.lua",
          path = "/Users/mtrajano/.local/share/nvim/lazy/one-small-step-for-vimkind/lua/osv/init.lua"
        }
      }, {
        column = 0,
        id = 4,
        line = 260,
        name = "highlight_win",
        source = {
          name = "@/Users/mtrajano/.local/share/nvim/lazy/todo-comments.nvim/lua/todo-comments/highlight.lua",
          path = "/Users/mtrajano/.local/share/nvim/lazy/todo-comments.nvim/lua/todo-comments/highlight.lua"
        }
      }, {
        column = 0,
        id = 5,
        line = 0,
        name = "main"
      } },
    totalFrames = 5
  },
  command = "stackTrace",
  request_seq = 5,
  seq = 5,
  success = true,
  type = "response"
}

[jbyuki]

Thanks, sounds a bit like I'm chasing my own shadow. I will push a fix shortly for that as well, adding an exception for stackframes inside the vim runtime.

[jbyuki]

@mtrajano could you have another go? This time all stack frames including osv internal stack frames are discarded. This should avoid the issue you described above, and hopefully fix the issue definitely.

[mtrajano]

@jbyuki Just gave it another try in the latest commit and it's still stopping in the same line, here's the new set of logs:

osv.log

{
  body = {
    stackFrames = { {
        column = 0,
        id = 1,
        line = 0,
        name = "main"
      }, {
        column = 0,
        id = 2,
        line = 0,
        name = "wait"
      }, {
        column = 0,
        id = 3,
        line = 1099,
        name = "Lua",
        source = {
          name = "@/Users/mtrajano/.local/share/nvim/lazy/one-small-step-for-vimkind/lua/osv/init.lua",
          path = "/Users/mtrajano/.local/share/nvim/lazy/one-small-step-for-vimkind/lua/osv/init.lua"
        }
      }, {
        column = 0,
        id = 4,
        line = 260,
        name = "highlight_win",
        source = {
          name = "@/Users/mtrajano/.local/share/nvim/lazy/todo-comments.nvim/lua/todo-comments/highlight.lua",
          path = "/Users/mtrajano/.local/share/nvim/lazy/todo-comments.nvim/lua/todo-comments/highlight.lua"
        }
      }, {
        column = 0,
        id = 5,
        line = 0,
        name = "main"
      } },
    totalFrames = 5
  },
  command = "stackTrace",
  request_seq = 5,
  seq = 5,
  success = true,
  type = "response"
}

[jbyuki]

Hey @mtrajano , I have pushed another commit. To explain a bit what I'm doing, I'm looking in the stack trace for the last stack frame which contains a reference to osv and pick the stack frame above that. To check if the stackframe is internal to osv, this is the relevant code. Maybe this is wrong, but I doubt it.

https://github.com/jbyuki/one-small-step-for-vimkind/blob/589dd96fb7b5458d7211ae93ed841bb32d5a5fee/lua/osv/init.lua#L648-L658

Please could you have another test?

[mtrajano]

Hi @jbyuki sorry for the late response, I think we are getting closer. This time it stopped on the correct breakpoint that I set but it was not able to step over it, I kept trying to hit step over and it would not move forward, attaching the log here again, it was just sequence of these lines in the log:

osv.log

{
  body = {
    reason = "step",
    threadId = 1
  },
  event = "stopped",
  seq = 22,
  type = "event"
}
{
  arguments = {
    threadId = 1
  },
  command = "stackTrace",
  seq = 20,
  type = "request"
}
{
  body = {
    stackFrames = { {
        column = 0,
        id = 7,
        line = 260,
        name = "highlight_win",
        source = {
          name = "@/Users/mtrajano/.local/share/nvim/lazy/todo-comments.nvim/lua/todo-comments/highlight.lua",
          path = "/Users/mtrajano/.local/share/nvim/lazy/todo-comments.nvim/lua/todo-comments/highlight.lua"
        }
      }, {
        column = 0,
        id = 8,
        line = 0,
        name = "main"
      } },
    totalFrames = 2
  },
  command = "stackTrace",
  request_seq = 20,
  seq = 23,
  success = true,
  type = "response"
}

[jbyuki]

Hi @mtrajano , thank you for your response. Indeed next/step is another part of the code where the offset was hardcoded. I changed so it's more adaptative but as always, as I cannot reproduce the bug on my side it's "hopefully" a fix. Please could you have another round?

[mtrajano]

Hey @jbyuki that's so weird I wonder why you're not able to repro. Here's the output of my system:

mtrajano@Mauricios-MBP nvim % uname -a
Darwin Mauricios-MBP 22.6.0 Darwin Kernel Version 22.6.0: Tue Nov  7 21:40:08 PST 2023; root:xnu-8796.141.3.702.9~2/RELEASE_ARM64_T6000 arm64
mtrajano@Mauricios-MBP nvim % nvim -v
NVIM v0.10.0
Build type: Release
LuaJIT 2.1.1716656478
Run "nvim -V1 -v" for more info

Latest commit:

In terms of testing it: I can't step over the current line and if I do it enough times the debug session eventually ends and an exception is thrown on the debugee which makes the editor unusable after the fact. Continue works to get past the current line by setting a break point in the next line but if you do step over a few times eventually the same thing happens. Attaching video of the issue happening. (<leader>dn is mapped to step over)

https://github.com/jbyuki/one-small-step-for-vimkind/assets/4333381/5d2a9b06-f187-4347-bccc-08623506945d

[jbyuki]

Hey @jbyuki that's so weird I wonder why you're not able to repro. Here's the output of my system:

To be honest I may have a slightly different setup so it's not a fair comparaison. I will try to mimic what you show in the video and see if I can somehow reproduce it. Otherwise, the issue will take a while to solve.

[jbyuki]

I tried to run the same setup and compare both logs.

I may have found the culprit, another hardcoded offset.

https://github.com/jbyuki/one-small-step-for-vimkind/blob/b3995f52ada9821aadd26e7c79647830c40786d1/lua/osv/init.lua#L1162-L1171

I will have to replace it with an adaptive offset again but this may slow down by a significant amount. But let's try, I will push a commit shortly.

[jbyuki]

@mtrajano Could you try once more? I'm almost 100% sure, this was causing an issue with stepping. The reason I cannot reproduce is because for some reason it goes less deep in the stack with my setup. I suspect it's because all vim runtimes are not precompiled in my neovim build (i.e. I can inspect inside vim runtime functions), where as by default it is. But I don't know for sure.

[jbyuki]

Just pushed another commit to fix all hardcoded offsets, this should fix the issue related to that in the future.

[mtrajano]

Hmmm unfortunately now it looks like the breakpoint is not hitting at all. This line is supposed to break on the WinScroll event

https://github.com/jbyuki/one-small-step-for-vimkind/assets/4333381/d22a634a-c5ac-4eec-9633-c1f3aaa51231

[jbyuki]

Wait, maybe I know why. Let me push another one.

[jbyuki]

Hoping this is the one 🙏🏻 I did a small mistake in the last commits that should be fixed now.

[mtrajano]

Success! The only issue left that I see is that the debug session does not seem to end, it seems to get stuck on the break point on continue but I guess terminating the session manually does work. This is still 100x more useful than the previous versions

https://github.com/jbyuki/one-small-step-for-vimkind/assets/4333381/b23690f1-e690-4889-90d3-ef3017289016

[jbyuki]

That was a tricky one, glad we got it in the end! I suspect this bug was quite widespread which basically meant osv was unusable for many, so it's really good to have solved this!!

In the last part of the video, is <leader>dc bound to continue and you're continously hitting the breakpoint ? I think this is expected behaviour. One way to get unstuck is just to remove the breakpoint, then continue. The debugging session should end once the debuggee is closed or require"osv".stop() is called in the debuggee. Otherwise yes it stays in a session as it's still attached to the debuggee.

[mtrajano]

Yes in the end of the video I was hitting continue and it was continuously hitting the breakpoint. I was under the impression that it would return control to the debuggee and the application would continue running normally, but that makes sense to me. It could also be the special case that this was a WinScrolled event that was continuously triggering. In any case I would say it makes sense to close the current issue. Thank you so much for taking the time in getting the issue fixed and creating this amazing plugin that is extremely useful!

[jbyuki]

Very nice! Thank you for your insightful comments that helped solved this complicated issue!