Exit codes

[benblank]

Exit codes are an important part of Notion's communication with external tools and a first-class part of the design of its user-friendly error messages. We need a strategy for ensuring they are used and communicated consistently.

Rendered

[dherman]

I think the main thing I'd like to see here is separating out the following cases:

• exit codes from notion
• exit codes from node
• exit codes from npm
• exit codes from yarn
• exit codes from npx and third-party executables

I would divide them up into three categories:

1. npm and yarn: This is where I'm most interested in getting value from exit codes, because when integrating into build/CI scripts you really want to be able to detect the main categories of errors. (Especially the network error category.) We don't control those tools but maybe we can engage with those organizations and see if there are ways they could communicate structured output (either via standard exit codes or execution modes (enabled via command-line flag or env var) that produce parseable output?) so we have a better ability to capture and categorize failures?
2. node, npx, and third-party executables: We have no control over these whatsoever, but if we have Notion-specific errors we still need to produce some sort of error code. How should we think about the fact that our exit codes and theirs have to share the same space of 255 values?
3. notion: We have total control here and can define whatever exit code scheme we want.

[benblank]

WRT npm and yarn, I don't like the idea of having our shims "rewrite" the tools' exit codes. Reaching out to those teams and trying to convince them of the value of good exit codes is another issue, but I think it's important that shims faithfully reproduce the behavior of the tools being shimmed, perhaps even to the point of producing a 0 exit code when the shimmed tool completes successfully even if Notion encounters errors (e.g. being unable to set up a new shim for a third-party executable after yarn installs it). If we start rewriting exit codes, we open up a whole new can of worms and the potential for misleading existing scripts which rely on those tools' current behavior.

Let me do some thinking and I'll try to update the RFC later today.

[benblank]

Apologies for the delay! I finally had some time to work on this and have expanded and partially rewritten the RFC.

[dherman]

Oh, one more thing: it looks like you've done (very thorough!) research on Unix/POSIX signals, but what about Windows conventions for exit codes?

[benblank]

Oh, one more thing: it looks like you've done (very thorough!) research on Unix/POSIX signals, but what about Windows conventions for exit codes?

I added a section addressing Windows exit codes and why I'm not proposing compatibility with them.

[dherman]

This is looking great and looks ready to merge to me. If no one raises any issues in the next couple days I'll merge it.