Moving `node-inspect` into core

[Trott]

Talked to a handful of @nodejs/ctc members (particularly @ofrobots) as well as @jkrems at Node Interactive today about this. A decision really needs to be made on what to do about the fact that Node.js CLI debugging is going to stop working in the foreseeable future. Unfortunately, there really doesn't seem to be consensus among the sampling of CTC folks I talked to.

Two obvious paths:

• Just remove it. People can use the inspector with a browser or they can install a userland package such as node-inspect

• Bring node-inspect into core. This has been discussed a little bit in the Diagnostics WG (see https://github.com/nodejs/diagnostics/issues/67) but I don't think it's really their call. @jkrems, the author and maintainer, has shown how it can be integrated with just a few additional lines of code. See https://github.com/nodejs/node/compare/master...jkrems:jk-node-inspect?expand=1

There are a subtleties and issues. I think it would be best if we make sure that both @ofrobots and @jkrems are at the meeting to fill in gaps, answer questions, etc.

I imagine we'll be canceling the meeting tomorrow, but hopefully we can talk about this at the meeting on Wednesday, December 7, 2016 at 16:00:00 UTC (8AM Pacific Time).

[sam-github]

/cc @bajtos

[bajtos]

@sam-github thank you for pinging me.

Honestly, I don't have much opinions here. At one side, I see the value of a built-in node inspect command that launches a CLI debugger, so that people don't have to rely on userland npm modules. On the other hand, I would not call 1500 lines of code in lib/_inspect.js "just a few additional lines of code" - there will be maintenance costs.

Here is a point that may be a decision-maker: how much activity do we expect in this new inspect component? Assuming the current node-inspect codebase is MVP-like and there is lots of space for improvement (even beyond what node debug provides): If node-inspect becomes a part of Node core, users on older Node versions (e.g. Node v6) will have to wait for new features (and possibly bugfixes too), since new development will be happening in newer Node versions only (v7, v8, etc.). If node-inspect remains a standalone package, then users can pull a newer version while still using their old Node runtime.

Isn't this similar to what we have already learned with (readable) stream(s)?

[sam-github]

I find that pretty compelling - does the debugger have to be in node core? If not, it shouldn't be, and as it communicates with node over a TCP protocol, I don't see why it has to be in core.

[jkrems]

@bajtos Yes, "a few additional lines of code" might be misleading. "Additional" as "in addition to the code that exists in node-inspect", not "additional to the code in node core".

There are definite advantages to all 3 options ("deprecate CLI debugging support", "provide an official standalone CLI debugger", and "provide a built-in CLI debug command in node itself"). Given the changes in the ecosystem since the original CLI debugger was added, "just drop CLI debugging from the feature list" is pretty tempting.

[jasnell]

I certainly agree with @sam-github on this. This could simply be an opportunity to remove that debugger from core in favor of the new --inspect tooling.

[wraithan]

I feel like this dismisses the fact that moving to using the chrome debugger protocol more directly was partially because node-inspector was userland, lagged behind on versions occasionally, etc. Basically making it unreliable for many folks. Now you're casting that on us CLI debugger users. I've dropped out of the node community to stay out of these arguments, but c'mon, you can't just kill CLI debugging completely and leave us with a userland solution that will likely have issues with time and drift of protocols and such.

[ofrobots]

@wraithan there is a qualitative difference between the node-inspector situation and here as there is a proper documented protocol on top of which, not only DevTools, but other tools such as Visual Studio Code, Firefox Debugger, WebStorm and others are built upon. CLI is just one more client.

[MylesBorins]

it is also worth mentioning that if we were to remove the CLI client from core the userland alternative would most likely be a project within the foundation, a very different situation from node-inspector

The idea to separate from core would primarily be to help encourage more maintainers and keep a healthy separation of concerns. It would still be possible for the project to shipped with core, even if it is maintained separately.

[wraithan]

@ofrobots Doesn't solve the problem. Just because there are multiple clients, doesn't mean that client/protocol drift isn't possible. node debug works regardless of version. This new CLI tool does not have that assertion.

Built in means it will always work with new versions, userland means it will update some time after a new version has come out or at least changed. It means keeping installs in tandem or making the client able to handle every protocol difference with time.

Finally the idea that this is required to make tools better is a fallacy at best. replpad got built back in the day, python has pdb built in and ipdb as a userland module. You can have good tools in userland without removing the more stripped down versions that are good to have when things hit the fan.

EDIT: I don't care if it is in the same source repo, I am asserting this needs to be distributed as part of node. If there is an issue for me to argue that without getting into "where the source is stored" debate. I'll go fight my fight there.

[sam-github]

@wraithan you do know that we are required to delete the node core CLI debugger, because the code behind it is being from v8? So, merely having code in core is not enough to say "it will work forever". The question is... do we then replace it wit a new one, or just let the userland CLI debugger continue to exist in userland? Where it was already created? And where (maybe) someone could even do some work to make it work on multiple protocols?

Its not often we have the chance to delete code from core, lets take it.

Btw, I think the comparison to the CLI debuggers many other scripting languages have is interesting, but I read it another way.

ruby/lua/python/perl/tcl/etc. authors all implement debug capability, and then as a kind of "proof it works", they implement one debugger on top, one that involves as little UI work as possible. So, they get a gdb like CLI debugger.

In node's case, v8 has the debug hooks, and it's primary author, google, implement a POC debugger as well - but they do it as a full UI in Chrome. So, Chrome's debugger is the debugger that has total commitment from the js/v8 language authors, and the CLI debugger in node is a historical relic.

[wraithan]

@sam-github I am aware that the current debugger cannot continue to live. I don't think I said "forever", I said the command works for the current version. Presumably the command would be removed, fixed or replaced rather than leaving it in, in a broken state. Right now we're in the "remove" discussion where I'd prefer it got "replaced", even under a new name to cause a new set of expectations.

I agree that those languages use their CLI debugger as more than what node may need it for. But the majority of folks who use those tools, are not using them because they are a proof of concept that the language can be debugged, they are using it to debug their applications and code.

[addaleax]

Right now we're in the "remove" discussion where I'd prefer it got "replaced", even under a new name to cause a new set of expectations.

The top comment in this issue specifically mentions the possible choice of bringing node-inspect into core, and as @TheAlphaNerd mentioned, it should be completely possible to ship it with Node itself.

[wraithan]

@addaleax Given this quote

Given the changes in the ecosystem since the original CLI debugger was added, "just drop CLI debugging from the feature list" is pretty tempting.

It seems like not distributing it was also being brought up here.

I did edit into my comment above (just after making it, and before you made this one) that I'll take this to another issue if there is another more appropriate one.

[Trott]

Something to bear in mind:

• If we remove the CLI debugger in version 8.0.0 and find out that is a terrible mistake, we can (at least in theory) add node-inspect or some other CLI debugger into core in version 8.1.0.

• If we ship a new debugger in version 8.0.0 and find out that is a terrible mistake, we will be required to support it for two and a half years (the duration of support for version 8.x.x).

That is not to say that there aren't compelling arguments for shipping a CLI debugger in version 8.0.0. Just adding this to the pile of things to consider. I'm actually very undecided on this issue at the moment. What I do know is that a decision needs to be made, and soon.

[wraithan]

I came into this incredibly hot headed because a feature I very strongly rely on (several times a day) may not be there when I need it.

Since this issue is about the source inclusion of the CLI debugger, and I don't care about that, I'm going to drop from this thread.

Regarding a CLI debugger being distributed with node itself, I've made my stance pretty clear why I think that's important. I don't think I can contribute anything of value beyond what I've already said here, so I'm going to bow completely out of the rest of this discussion, since whatever the decision that is made will have that opinion factored in along side of the maintenance cost and personal biases which may agree or disagree with my personal bias.

[Fishrock123]

The debugging story is already bad enough, we already shipped node debug before, so I don't see why (or frankly how) we could not ship an updated version (even if it is usable by node inspect instead).

[bajtos]

Built in means it will always work with new versions, userland means it will update some time after a new version has come out or at least changed. It means keeping installs in tandem or making the client able to handle every protocol difference with time.

I agree with this comment, we have been bitten by DevTools changes in node-inspector a lot.

However, I think (hope?) the situation should be much different (and easier) for node inspect.

In Node Inspector, we were re-implementing the DevTools server and had to keep up with the ever-changing DevTools UI. Whenever the UI added a new feature, or started using a different DevTools command, we had to jump in and extend DevTools protocol bridge in Node Inspector too.

On the other hand in node inspect, we are merely consuming DevTools protocol provided by V8/Node.js core. As long as DevTools protocol stays (mostly) backwards compatible, supporting multiple Node.js version in the same CLI debugger should be very easy.

https://github.com/nodejs/CTC/issues/40#issuecomment-264985907

it is also worth mentioning that if we were to remove the CLI client from core the userland alternative would most likely be a project within the foundation, a very different situation from node-inspector The idea to separate from core would primarily be to help encourage more maintainers and keep a healthy separation of concerns. It would still be possible for the project to shipped with core, even if it is maintained separately.

To me personally, this looks like a good approach to take. Keep the CLI debugger as a standalone project/package published regularly to npm, so that people can update to a newer version if they like. Ship a copy (a snapshot) of this package as part of Node.js, so that a working (even if outdated) CLI debugger is always there when needed. In my view, this is following the existing (and proven?) approach we use for npm.

[Fishrock123]

@bajtos I'm unclear how that would work, do you mean node inspect would launch a bundled version, or do you mean the installers would globally install a node-inspect node module or something? Tarballs would just ship it in a node_modules...?

Could you be a bit more specific?

[sam-github]

@Fishrock123 he means that like npm, it would be a project with its own nodejs/debugger repo, and would be developed there, and npm published to npmjs.org, but it would also be vendored into nodejs/node:deps/, as npm is. But vendored deps (like npm) can't be easily changed, so it would be possible to update to the latest when your debugger has bugs (sigh), by doing an npm install.

[bajtos]

Ha, thank you @sam-github for commenting, you explained it really well.

Below is my (half-finished) comment that I was crafting in the meantime.

@Fishrock123

I'm unclear how that would work, do you mean node inspect would launch a bundled version, or do you mean the installers would globally install a node-inspect node module or something? Tarballs would just ship it in a node_modules...?

I didn't thought it that far, to be honest. How is npm shipped now? Can we reuse the same approach? What would be the downsides of shipping node-inspect similarly to how npm is shipped?

I think the high-level end-user experience is what really matters here:

• I install Node.js and get a working CLI debugger that's been fully tested with this particular Node.js version.
• I can npm install a newer version of the CLI debugger if I like to, with the caveat that the latest debugger version may not fully support my old Node.js version.

I think I am starting to understand your concerns now. Because node binary is a self-contained, if we want node inspect to work correctly, then node-inspect source code has to be compiled into node binary. In which case we cannot use the approach of how npm is bundled in Node.js installer, because npm is invoked as a user-land script, not a built-in node command. Also one could end up with two debuggers, node inspect and node-inspect, where each debugger behaves differently. This may be either confusing, or a good thing, depending on the point of view.

Sorry if I created more questions than answers!

[refack]

So the solution is nodejs/node#10187 which closes nodejs/node#7266? And we get a (deprecated) CLI debugger bundled for a little longer? mark this issue as closed?

[jkrems]

And we get a (deprecated) CLI debugger bundled for a little longer?

I don't think there's a plan to deprecate the CLI debugger. The only thing that's deprecated (and will be removed) is the old protocol.

mark this issue as closed?

👍

[refack]

I ment deprece node debug in favor of node inspect... nodejs/node#10276

[joshgav]

@refack

deprecate node debug in favor of node inspect

We'll have to decide if node debug should continue to work a la or instead of node inspect, that discussion should continue in https://github.com/nodejs/node/pull/10187.

I'm about to split https://github.com/nodejs/node/pull/10276 into two so that deprecation of node --debug can land soon, and we can address node debug separately.

[joshgav]

The TSC approved bringing the node-inspect repo into the Foundation in https://github.com/nodejs/TSC/issues/190#issuecomment-272546891, and the CTC has approved that as well, so the repo has now been transferred: https://github.com/nodejs/node-inspect

This thread is about whether to include node-inspect in core, i.e. nodejs/node. I believe we have consensus that it should be included somehow.

The current suggestion on how to include it is reflected by the current state of https://github.com/nodejs/node/pull/10187, and I believe is:

1. include in deps folder
2. integrate into command-line as node inspect and/or node debug.

[refack]

So this thread should be closed?

[jasnell]

This has been done. Closing