errors: add support for wrapping multiple errors

[neild]

For the most recent version of this proposal, see: https://github.com/golang/go/issues/53435#issuecomment-1191752789 below.

---

This is a variation on the rejected proposal #47811 (and perhaps that proposal should just be reopened), and an expansion on a comment in it.

Background

Since Go 1.13, an error may wrap another by providing an Unwrap method returning the wrapped error. The errors.Is and errors.As functions operate on chains of wrapped errors.

A common request is for a way to combine a list of errors into a single error.

Proposal

An error wraps multiple errors if its type has the method

Unwrap() []error

Reusing the name Unwrap avoids ambiguity with the existing singular Unwrap method. Returning a 0-length list from Unwrap means the error doesn't wrap anything. Callers must not modify the list returned by Unwrap. The list returned by Unwrap must not contain any nil errors.

We replace the term "error chain" with "error tree".

The errors.Is and errors.As functions are updated to unwrap multiple errors. Is reports a match if any error in the tree matches. As finds the first matching error in a preorder traversal of the tree.

The errors.Join function provides a simple implementation of a multierr. It does not flatten errors.

// Join returns an error that wraps the given errors.
// Any nil error values are discarded.
// The error formats as the text of the given errors, separated by sep.
// Join returns nil if errs contains no non-nil values.
func Join(sep string, errs ...error) error

The fmt.Errorf function permits multiple instances of the %w formatting verb.

The errors.Split function retrieves the original errors from a combined error.

// Split returns the result of calling the Unwrap method on err,
// if err's type contains an Unwrap method returning []error.
// Otherwise, Split returns nil.
func Split(err error) []error

The errors.Unwrap function is unaffected: It returns nil when called on an error with an Unwrap() []error method.

Questions

Prior proposals have been declined on the grounds that this functionality can be implemented outside the standard library, and there was no good single answer to several important questions.

Why should this be in the standard library?

This proposal adds something which cannot be provided outside the standard library: Direct support for error trees in errors.Is and errors.As. Existing combining errors operate by providing Is and As methods which inspect the contained errors, requiring each implementation to duplicate this logic, possibly in incompatible ways. This is best handled in errors.Is and errors.As, for the same reason those functions handle singular unwrapping.

In addition, this proposal provides a common method for the ecosystem to use to represent combined errors, permitting interoperation between third-party implementations.

How are multiple errors formatted?

A principle of the errors package is that error formatting is up to the user. This proposal upholds that principle: The errors.Join function combines error text with a user-provided separator, and fmt.Errorf wraps multiple errors in a user-defined layout. If users have other formatting requirements, they can still create their own error implementations.

How do Is and As interact with combined errors?

Every major multierror package that I looked at (see "Prior art" below) implements the same behavior for Is and As: Is reports true if any error in the combined error matches, and As returns the first matching error. This proposal follows common practice.

Does creating a combined error flatten combined errors in the input?

The errors.Join function does not flatten errors. This is simple and comprehensible. Third-party packages can easily provide flattening if desired.

Should Split unwrap errors that wrap a single error?

The errors.Split function could call the single-wrapping Unwrap() error method when present, converting a non-nil result into a single-element slice. This would allow traversing an error tree with only calls to Split.

This might allow for a small improvement in the convenience of code which manually traverses an error tree, but it is rare for programs to manually traverse error chains today. Keeping Split as the inverse of Join is simpler.

Why does the name of the Split function not match the Unwrap method it calls?

Giving the single- and multiple-error wrapping methods the same name neatly avoids any questions of how to handle errors that implement both.

Split is a natural name for the function that undoes a Join.

While we could call the method Split, or the function UnwrapMultiple, or some variation on these options, the benefits of the above points outweigh the value in aligning the method name with the function name.

Prior art

There have been several previous proposals to add some form of combining error, including:

https://go.dev/issue/47811: add Errors as a standard way to represent multiple errors as a single error https://go.dev/issue/48831: add NewJoinedErrors https://go.dev/issue/20984: composite errors https://go.dev/issue/52607: add With(err, other error) error fmt.Errorf("%w: %w", err1, err2) is largely equivalent to With(err1, err2).

Credit to @jimmyfrasche for suggesting the method name Unwrap.

There are many implementations of combining errors in the world, including:

https://pkg.go.dev/github.com/hashicorp/go-multierror (8720 imports) https://pkg.go.dev/go.uber.org/multierr (1513 imports) https://pkg.go.dev/tailscale.com/util/multierr (2 imports)

[DeedleFake]

How does walking up the tree work in errors.Is() and errors.As()? Is it depth-first, i.e. Unwrap() to []error and then fully walk each resulting tree, or is it breadth-first, i.e. Unwrap() to []error and then do each of those before unwrapping them and doing the next set. I assume that it's depth-first, as that's simpler, but breadth-first could also make sense. It would also be possible to convert from a default of breadth-first to a depth-first by implementing custom unwrapping logic, but going the other way could be difficult.

[neild]

How does walking up the tree work in errors.Is() and errors.As()?

Depth-first. Unwrap to an []error and walk each error in the list in turn. This is what every existing multierr implementation I looked at does.

[earthboundkid]

This would also subsume #50819 and I imagine any other similar proposals.

Maybe it should wait for a generic iteration standard, but should there be some errors.Each function that goes through each link in the chain? My suspicion is there’s not much to do besides As-ing and Is-ing, but it does seem like an absence, since there will be some internal function that works like that, which could potentially be exported.

[neild]

Is and As are separate implementations for efficiency reasons, so there's no internal Each function. This proposal does make the case for an exported Each or similar stronger, since iteration becomes more complicated.

We should probably wait and see what happens with generic iteration, however.

[balasanjay]

Does Unwrap return a reference to the internal slice in the error? If so, that would make the internal slice reference externally mutable, which seems unfortunate (but not a dealbreaker).

If not, and it returns a copy, then it would allocate, which also seems unfortunate (but probably not a dealbreaker).

[neild]

Does Unwrap return a reference to the internal slice in the error?

Unspecified, but the contract for it says that the caller must not modify the slice, so it may.

Perhaps Split should copy the result into a new slice. This would leave Is and As potentially allocation-free, while adding a layer of safety (and an allocation) to the relatively unusual case of manually inspecting the individual errors.

[josharian]

This strikes me as a significant step forward in figuring out the multi-error question. Thank you!

The only part of this that doesn't seem entirely spot-on is errors.Split.

What is the use case for errors.Split?

Figuring errors.Split would be used like errors.Unwrap, I grepped through the code sitting around on my laptop and found only a single use of errors.Unwrap. It was used to check recursively whether any error in the chain satisfied a predicate. It would be interesting to analyze other uses in a larger corpus.

Walking an arbitrary error tree using errors.Split and errors.Unwrap will be annoying. I wonder whether a callback-based errors.Walk that walks the entire error tree would be a better API. There are a lot of details to specify for such an API, but with a bit of care, it could be more powerful and flexible than errors.Split. It could be used to implement errors.Is and errors.As. And it sidesteps questions about ownership and modification of returned error slices.

(The details about errors.Walk include things like: Does it call back for all errors, or only leaf errors? Does it use a single callback like io/fs.WalkDir or separate pre- and post- callbacks like golang.org/x/tools/ast/astutil.Apply? Does it support early stopping?)

On the topic of slice copies, a minor question. Does errors.Join make a copy of the variadic arg? I'd argue for yes, and ameliorate the pain a bit by adding a small default backing store to the struct. Aliasing bugs are miserable.

[earthboundkid]

I find that the main reason I unwrap my own multierrors is so that I can report them separately to my logging service or display them as a list in a CLI. The original xerrors formatting proposal was dropped for being too complicated but I think that an easy to use error walking mechanism might make it easy to let users figure out how they want to display and format all the suberrors on their own.

[rsc]

This proposal has been added to the active column of the proposals project and will now be reviewed at the weekly proposal review meetings. — rsc for the proposal review group

[hopehook]

If I hadn't read the comments, I might have thought that after the "Join" error, I could use "Split" to parse it out.

"errors.Split" does the same thing as "errors.Unwrap", but deals with multiple errors. So instead of "Split" which is the opposite of "Join", it should be "UnwrapMultiple" or "Walk" which is more appropriate.

[hopehook]

The errors.Unwrap function is unaffected: It returns nil when called on an error with an Unwrap() []error

Since there is no reaction to multiple error conditions, the naming should also be differentiated, such as ”UnwrapMultiple() []error “.

[neild]

What is the use case for errors.Split?

We need to provide some reasonably convenient way to get at the underlying errors of a multierr. Perhaps Split isn't the right API here (although I still feel pretty good about it), but we need something.

I wonder whether a callback-based errors.Walk that walks the entire error tree would be a better API.

Perhaps there's a case for such an API, but I think it's distinct from this proposal. Accessing the component errors of a multierr is different from a full tree walk.

Does errors.Join make a copy of the variadic arg?

Yes.

[jimmyfrasche]

I'm 100% on

• defining Unwrap() []error
• changing Is/As to support the new Unwrap
• updating fmt.Errorf to allow multiple %w

If those were the only things accepted I'd be happy and it would be more than enough to allow external multierrors to interoperate with each other and std.

I do think there 100% does need to be a tree walking API—but it's too early to decide what that looks like. Once the protocol above is enshrined in errors, it's simple enough to experiment with that outside std.

I'm not 100% sold on the name for Split but there should probably be such a func in std, regardless of name, to pair with Unwrap. I don't think it would be much of an impediment if this weren't included, however.

I'm not really sold on Join. I see the utility of including a basic multierror in std, but just gluing the errors together with a sep seems like it could get messy when multierrors get composed. I think, like a tree walking API, it should be left out and experimentation allowed to continue outside std for now.

[josharian]

We need to provide some reasonably convenient way to get at the underlying errors of a multierr.

I'm not so sure that we do.

For uses (testing?) in which you really specifically want to unwrap from an error that has an Unwrap() [] method, you can use a type assertion. (errors.Unwrap is less than 10 LOC, and consists entirely of a type assertion.)

But I suspect that almost nobody wants specifically to unwrap a multi-error exactly one layer deep. Rather they want to ask the general question "what errors are in this tree?". So I think that's the right form of API to add. And I think it'll get created internally anyway to implement Is and As.

Maybe this proposal should simply omit Split?

[neild]

Rather they want to ask the general question "what errors are in this tree?".

The fact that errors.Unwrap is so seldom used indicates to me that people don't want to ask this question. They use Is and As to ask "is this error in the tree?" or "is there an error of this type in the tree?" instead.

I'm not certain how much of a use case there is for a tree walk that doesn't preserve the structure of the tree. It would be nice to see real-world examples of this in practice.

[josharian]

I grepping in my own dependencies and grabbed all the uses. I ignored the standard library, static analyzers, linters, and then pulled all the uses that were not in tests. Here they are:

x/tools uses errors.Unwrap to get to the innermost error, without regard for the structure of what is above it. I suspect that this could would be better served by being able to evaluate a predicate on every error in the tree.

aws-sdk-go uses errors.Unwrap to recursively evaluate a predicate on every error in the tree, with early stopping.

containerd uses errors.Unwrap in a way that I find confusing and might be buggy. It uses it to peel back exactly one layer of wrapping and then compares this to a particular error using string matching. This is fragile (if another wrapping layer gets introduced) and would probably be better served by a way to check every error in the tree.

containerd uses errors.Unwrap again to peel back exactly one layer of wrapping. It looks like errors.As would be a better fit here; I don't see any reason why exactly one layer is the right answer here.

jwt uses errors.Unwrap to implement Is. In this case, errors.Unwrap is unnecessary, because e is known to have a concrete type that implements Unwrap; the code could call e.Unwrap directly instead.

I then grabbed two uses in tests at random:

errwrap uses errors.Unwrap specifically to test that it is compatible with how the standard library defines errors.Unwrap. It has no actual use for errors.Unwrap or opinion about its behavior.

pgconn uses errors.Unwrap to test its own error's compability with package errors. It looks like it would be equally well served by errors.As.

So far, I see no evidence that anyone cares much about the structure of the error tree. It looks to me like they are all: misusing the API (evidence that the API is wrong); using the wrong API (evidence that the API is wrong); using the API pointlessly; or using the API to evaluate a predicate "anywhere along the error chain".

[neild]

That's useful data; thanks.

We don't have error trees right now; we have an error chain. We can't tell whether people will care about the structure of trees by looking at how they work with linear chains.

It'd be interesting to look at use of existing multierr packages to see how often multierrs are converted back to lists of errors.

[josharian]

It'd be interesting to look at use of existing multierr packages to see how often multierrs are converted back to lists of errors.

I agree. Can I leave that for you to do? I've exceeded my time allotment for this for a while. :)

[hopehook]

I'm 100% on

• defining Unwrap() []error
• changing Is/As to support the new Unwrap
• updating fmt.Errorf to allow multiple %w

If those were the only things accepted I'd be happy and it would be more than enough to allow external multierrors to interoperate with each other and std.

I do think there 100% does need to be a tree walking API—but it's too early to decide what that looks like. Once the protocol above is enshrined in errors, it's simple enough to experiment with that outside std.

I agree with you very much, we should focus on the original core API supporting the multiple error model first, and then expose the complexity to the user. Before that, we can experiment with Join, Split, Walk, etc.

Most importantly, decide the definition of Unwrap() []error.

[neild]

I agree. Can I leave that for you to do? I've exceeded my time allotment for this for a while. :)

I'm going to go look, but probably not until next week. Had a few days of meetings, and I'm now exhausted and behind on everything. :)

[bn-jbowers]

Given the size of these error trees, where even "a dozen" would be huge, I would suggest the solution to iteration is simply to define that the traversal used by the rest of the library will be used to return a []error populated in the traversal order used by the rest of the code, or a simple tree struct. Such a small set of values doesn't rate having the iteration proposal placed as a blocker in front of it.

It's rarely used functionality (as observed above for Unwrap in general), on the error path (not the happy path where performance is generally a bigger deal), for a rarely-used use case where Is and As is not enough. Code that is generating millions or billions of complex error trees per second that it then somehow needs to deal with in a complicated manner is too broken to be saved by even zero-performance error tree iteration.

[BenjamenMeyer]

I'd like to help encourage my team to use errors.Is (or evens errors.As) for testing instead of using text-based comparisons. While researching to provide best practices I created the following code for use in the Golang Playground:

https://gist.github.com/BenjamenMeyer/ef9926913dcc3b165da8f25a459442a9

I was quite surprised that errors.Unwrap would only extract the first layer (may be I'm doing something wrong?) but then came across this proposal. Honestly, IMHO most are looking to be able to use the errors.is and errors.As APIs to test a full error chain reliably instead of having to decode a string, which could be unreliable.

Love the proposal here but would certainly be happy with something that would allow errors.Is to detect any given error is part of the provided error when built up from several in a row - e.g returning an error up from a lower level in the code to a higher level several times and being able to see the lower error is there.

NOTE: The code in the gist is just a sample for this explicit behavior. Consider the list of errors representing the layers of the code with the singular error what the code actually sees in the tests; yet it fails to be able to detect the lower level errors.

HTH

Follow-up: This seems to be something that has gone back to 1.13-betas with properly wrapping and detecting the various layered errors: https://groups.google.com/g/golang-nuts/c/SwKZ-qJmZl0?pli=1

[neild]

I've compiled some data on use of splitting multierrors created with go.uber.org/multierr back inmto the original list of errors.

I started with a list of 1976 modules known to pkg.go.dev which import go.uber.org/multierr. I populated my local module cache with this set by running go get $MOD@latest for each one. I then used grep to build a list of .go files in my module cache that imported this package, and filtered this down to only files in the latest version of each module when there were duplicates. There's probably a more elegant way to do this, but it worked for me.

This gave me 5981 unique files that import go.uber.org/multierr.

Of those, 182 call multierr.Error to convert an error into a []error.

The following is a set of links to uses of multierr.Error calls, with a few deleted packages removed. Many of these look to be forks of the same original code; I haven't made any effort to deduplicate these.

expand

- https://github.com/berty/berty/blob/v2.420.0/go/framework/bertybridge/bridge.go#L270 - https://github.com/berty/berty/blob/v2.420.0/go/pkg/bertymessenger/api.go#L323 - https://github.com/berty/berty/blob/v2.420.0/go/pkg/bertymessenger/notifiee_test.go#L28 - https://github.com/berty/berty/blob/v2.420.0/go/pkg/bertymessenger/notifiee_test.go#L60 - https://github.com/berty/berty/blob/v2.420.0/go/pkg/bertymessenger/notifiee_test.go#L108 - https://github.com/berty/berty/blob/v2.420.0/go/pkg/bertyprotocol/api_debug.go#L224 - https://github.com/emperror/errors/blob/v0.8.1/errors_multi.go#L68 - https://github.com/AndreMouche/br/blob/v4.0.11/pkg/restore/import.go#L249 - https://github.com/AndreMouche/br/blob/v4.0.11/pkg/restore/backoff_test.go#L75 - https://github.com/AndreMouche/br/blob/v4.0.11/pkg/restore/backoff_test.go#L92 - https://github.com/AndreMouche/br/blob/v4.0.11/pkg/restore/backoff_test.go#L105 - https://github.com/DigitalChinaOpenSource/TiDB-for-PostgreSQL/blob/8fe2e17f2fba/br/pkg/utils/backoff_test.go#L73 - https://github.com/DigitalChinaOpenSource/TiDB-for-PostgreSQL/blob/8fe2e17f2fba/br/pkg/utils/backoff_test.go#L90 - https://github.com/DigitalChinaOpenSource/TiDB-for-PostgreSQL/blob/8fe2e17f2fba/br/pkg/utils/backoff_test.go#L103 - https://github.com/DigitalChinaOpenSource/TiDB-for-PostgreSQL/blob/8fe2e17f2fba/br/pkg/utils/backoff_test.go#L126 - https://github.com/DigitalChinaOpenSource/TiDB-for-PostgreSQL/blob/8fe2e17f2fba/br/pkg/restore/import.go#L341 - https://github.com/DigitalChinaOpenSource/TiDB-for-PostgreSQL/blob/8fe2e17f2fba/br/pkg/restore/systable_restore.go#L102 - https://github.com/DigitalChinaOpenSource/dcbr/blob/v4.0.12/pkg/restore/import.go#L249 - https://github.com/DigitalChinaOpenSource/dcbr/blob/v4.0.12/pkg/restore/backoff_test.go#L75 - https://github.com/DigitalChinaOpenSource/dcbr/blob/v4.0.12/pkg/restore/backoff_test.go#L92 - https://github.com/DigitalChinaOpenSource/dcbr/blob/v4.0.12/pkg/restore/backoff_test.go#L105 - https://github.com/JitendraKSahu/beats/v7/blob/v7.0.68/x-pack/filebeat/processors/decode_cef/decode_cef.go#L96 - https://github.com/JmPotato/br/blob/c7f93829ef43/pkg/restore/import.go#L331 - https://github.com/JmPotato/br/blob/c7f93829ef43/pkg/restore/backoff_test.go#L75 - https://github.com/JmPotato/br/blob/c7f93829ef43/pkg/restore/backoff_test.go#L92 - https://github.com/JmPotato/br/blob/c7f93829ef43/pkg/restore/backoff_test.go#L105 - https://github.com/JmPotato/br/blob/c7f93829ef43/pkg/restore/systable_restore.go#L103 - https://github.com/LuoHongLiang0921/kuaigo/blob/v1.0.12/pkg/util/kgo/serial_test.go#L42 - https://github.com/Mirantis/mcc-api/blob/c51dca782049/pkg/errors/helper.go#L89 - https://github.com/Orion7r/br/blob/v5.1.0/pkg/restore/import.go#L331 - https://github.com/Orion7r/br/blob/v5.1.0/pkg/restore/backoff_test.go#L75 - https://github.com/Orion7r/br/blob/v5.1.0/pkg/restore/backoff_test.go#L92 - https://github.com/Orion7r/br/blob/v5.1.0/pkg/restore/backoff_test.go#L105 - https://github.com/Orion7r/br/blob/v5.1.0/pkg/restore/systable_restore.go#L103 - https://github.com/TencentBlueKing/beats/blob/v7.6.2/x-pack/filebeat/processors/decode_cef/decode_cef.go#L99 - https://github.com/abulo/ratel/v3/blob/v3.1.2/goroutine/serial_test.go#L41 - https://github.com/achillesss/jupiter/blob/v0.2.5/pkg/util/xgo/serial_test.go#L54 - https://github.com/agilebits/beats/blob/v7.6.2/x-pack/filebeat/processors/decode_cef/decode_cef.go#L99 - https://github.com/akityo/notify/blob/v1.1.0/pkg/providers/providers.go#L114 - https://github.com/aleksmaus/beats/blob/v7.6.2/x-pack/filebeat/processors/decode_cef/decode_cef.go#L99 - https://github.com/andreiubr/yab/blob/v0.17.0/plugin/plugin_test.go#L65 - https://github.com/andreiubr/yab/blob/v0.17.0/plugin/plugin_test.go#L81 - https://github.com/andremouche/br/blob/v4.0.11/pkg/restore/import.go#L249 - https://github.com/andremouche/br/blob/v4.0.11/pkg/restore/backoff_test.go#L75 - https://github.com/andremouche/br/blob/v4.0.11/pkg/restore/backoff_test.go#L92 - https://github.com/andremouche/br/blob/v4.0.11/pkg/restore/backoff_test.go#L105 - https://github.com/anexia-it/geodbtools/blob/v1.0.1/cmd/geodbtool/cmd_convert.go#L146 - https://github.com/anexia-it/geodbtools/blob/v1.0.1/verify_test.go#L178 - https://github.com/anexia-it/geodbtools/blob/v1.0.1/verify_test.go#L262 - https://github.com/anexia-it/geodbtools/blob/v1.0.1/verify_test.go#L299 - https://github.com/ansoni/beats/blob/v7.5.2/x-pack/filebeat/processors/decode_cef/decode_cef.go#L99 - https://github.com/apigee/istio-mixer-adapter/blob/dcd54df6a221/apigee-istio/cmd/provision/provision.go#L268 - https://github.com/appointy/fx/blob/v1.9.0/app_test.go#L469 - https://github.com/artefactual-labs/amflow/blob/v0.1.0-beta.5/cmd/cmd.go#L108 - https://github.com/asalih/beats/blob/v7.6.2/x-pack/filebeat/processors/decode_cef/decode_cef.go#L99 - https://github.com/aws/amazon-ec2-instance-selector/v2/blob/v2.3.2/pkg/ec2pricing/ec2pricing.go#L97 - https://github.com/aws/karpenter/blob/v0.13.2/pkg/controllers/provisioning/provisioner.go#L100 - https://github.com/axw/beats/blob/v7.6.0/x-pack/filebeat/processors/decode_cef/decode_cef.go#L99 - https://github.com/bevgene/go-currency-rate/blob/749d40c8f5e4/app/validations/currency_converter.go#L52 - https://github.com/bhbosman/fx/blob/v1.13.0/app_test.go#L531 - https://github.com/bit2pixel/fx/blob/v1.13.2/app_test.go#L543 - https://github.com/bmartynov/fx/blob/v1.6.0/app_test.go#L343 - https://github.com/bufbuild/buf/blob/v1.6.0/private/buf/bufgen/generator.go#L223 - https://github.com/build-security/beats/blob/v7.6.2/x-pack/filebeat/processors/decode_cef/decode_cef.go#L99 - https://github.com/chapsuk/fx/blob/v1.8.0/app_test.go#L384 - https://github.com/chenjianmei111/go-ds-versioning/blob/v0.2.0/internal/migrate/migrate_test.go#L166 - https://github.com/colstuwjx/beats/blob/v7.6.2/x-pack/filebeat/processors/decode_cef/decode_cef.go#L99 - https://github.com/cxt90730/br/blob/v4.0.13/pkg/restore/import.go#L250 - https://github.com/cxt90730/br/blob/v4.0.13/pkg/restore/backoff_test.go#L75 - https://github.com/cxt90730/br/blob/v4.0.13/pkg/restore/backoff_test.go#L92 - https://github.com/cxt90730/br/blob/v4.0.13/pkg/restore/backoff_test.go#L105 - https://github.com/digitalchinaopensource/dcbr/blob/v4.0.11/pkg/restore/import.go#L249 - https://github.com/digitalchinaopensource/dcbr/blob/v4.0.11/pkg/restore/backoff_test.go#L75 - https://github.com/digitalchinaopensource/dcbr/blob/v4.0.11/pkg/restore/backoff_test.go#L92 - https://github.com/digitalchinaopensource/dcbr/blob/v4.0.11/pkg/restore/backoff_test.go#L105 - https://github.com/disksing/br/blob/3d29958e0b98/pkg/restore/import.go#L254 - https://github.com/disksing/br/blob/3d29958e0b98/pkg/restore/backoff_test.go#L75 - https://github.com/disksing/br/blob/3d29958e0b98/pkg/restore/backoff_test.go#L92 - https://github.com/disksing/br/blob/3d29958e0b98/pkg/restore/backoff_test.go#L105 - https://github.com/dogmatiq/projectionkit/blob/v0.6.3/sqlprojection/select_test.go#L23 - https://github.com/dogmatiq/verity/blob/v0.1.0/persistence/sqlpersistence/provider_test.go#L140 - https://github.com/douyu/jupiter/blob/v0.4.9/pkg/util/xgo/serial_test.go#L54 - https://github.com/elastic/beats/v7/blob/v7.17.5/x-pack/filebeat/processors/decode_cef/decode_cef.go#L96 - https://github.com/elastic/beats/blob/v7.6.2/x-pack/filebeat/processors/decode_cef/decode_cef.go#L99 - https://github.com/enmasseproject/enmasse/blob/e4e541c29d0d/pkg/util/errors.go#L33 - https://github.com/erenming/beats/blob/v7.6.2/x-pack/filebeat/processors/decode_cef/decode_cef.go#L99 - https://github.com/eric4545/beats/blob/v7.6.2/x-pack/filebeat/processors/decode_cef/decode_cef.go#L99 - https://github.com/eyugod/beats/blob/v7.6.0/x-pack/filebeat/processors/decode_cef/decode_cef.go#L99 - https://github.com/filecoin-project/go-ds-versioning/blob/v0.1.2/internal/migrate/migrate_test.go#L166 - https://github.com/fionera/fx/blob/v1.10.0/app_test.go#L519 - https://github.com/foolishnoob/go-xkratos/blob/ce3a2b72a290/util/xgo/serial_test.go#L54 - https://github.com/freemen-app/configuro/blob/v0.0.5/error.go#L56 - https://github.com/freemen-app/configuro/blob/v0.0.5/configuro_test.go#L799 - https://github.com/freemen-app/configuro/blob/v0.0.5/configuro_test.go#L855 - https://github.com/freemen-app/configuro/blob/v0.0.5/configuro_test.go#L907 - https://github.com/freemen-app/configuro/blob/v0.0.5/validate.go#L32 - https://github.com/gee-m/errors/blob/v0.8.0/errors_multi.go#L68 - https://github.com/glorv/br/blob/v4.0.11-hotfix/pkg/restore/import.go#L249 - https://github.com/glorv/br/blob/v4.0.11-hotfix/pkg/restore/backoff_test.go#L75 - https://github.com/glorv/br/blob/v4.0.11-hotfix/pkg/restore/backoff_test.go#L92 - https://github.com/glorv/br/blob/v4.0.11-hotfix/pkg/restore/backoff_test.go#L105 - https://github.com/grongor/go-bootstrap/blob/bd6f409ef1ec/config.go#L114 - https://github.com/hacdias/fintracts/fintracts/blob/88bd814911b0/cmd/fintracts/utils.go#L27 - https://github.com/hahawangxv/kevin/blob/d6ac8213b563/pkg/util/xgo/serial_test.go#L54 - https://github.com/hrak/beats/blob/v7.5.2/x-pack/filebeat/processors/decode_cef/decode_cef.go#L99 - https://github.com/hu177/br/blob/v5.3.0/pkg/restore/import.go#L331 - https://github.com/hu177/br/blob/v5.3.0/pkg/restore/backoff_test.go#L75 - https://github.com/hu177/br/blob/v5.3.0/pkg/restore/backoff_test.go#L92 - https://github.com/hu177/br/blob/v5.3.0/pkg/restore/backoff_test.go#L105 - https://github.com/hu177/br/blob/v5.3.0/pkg/restore/systable_restore.go#L103 - https://github.com/hukaixuan/beats/blob/v7.6.2/x-pack/filebeat/processors/decode_cef/decode_cef.go#L99 - https://github.com/hxzhouh/zrpc/blob/ca5b6c233c9b/pkg/utils/xgo/serial_test.go#L54 - https://github.com/iZettle/maperr/v4/blob/v4.3.5/list.go#L43 - https://github.com/iZettle/maperr/v4/blob/v4.3.5/hashable.go#L31 - https://github.com/iZettle/maperr/v4/blob/v4.3.5/ignore_list.go#L34 - https://github.com/iZettle/maperr/v4/blob/v4.3.5/maperr.go#L142 - https://github.com/iZettle/maperr/v4/blob/v4.3.5/maperr.go#L153 - https://github.com/iZettle/maperr/v4/blob/v4.3.5/maperr.go#L169 - https://github.com/influxdata/influxdb/v2/blob/v2.3.0/measurement_schema_test.go#L271 - https://github.com/ipfs/ipfs-cluster/blob/v1.0.2/adder/util.go#L95 - https://github.com/jeffbean/fx/blob/v1.7.0/app_test.go#L361 - https://github.com/jmpotato/br/blob/c7f93829ef43/pkg/restore/import.go#L331 - https://github.com/jmpotato/br/blob/c7f93829ef43/pkg/restore/backoff_test.go#L75 - https://github.com/jmpotato/br/blob/c7f93829ef43/pkg/restore/backoff_test.go#L92 - https://github.com/jmpotato/br/blob/c7f93829ef43/pkg/restore/backoff_test.go#L105 - https://github.com/jmpotato/br/blob/c7f93829ef43/pkg/restore/systable_restore.go#L103 - https://github.com/liov/tiga/blob/1f4ee57034ff/utils/errors/multierr/multierr.go#L41 - https://github.com/liov/tiga/blob/1f4ee57034ff/utils/errors/multierr/multierr.go#L189 - https://github.com/lunarway/release-manager/blob/v0.22.3/cmd/server/http/error.go#L42 - https://github.com/lunarway/release-manager/blob/v0.22.3/internal/flow/new_artifact.go#L115 - https://github.com/marc-gr/beats/blob/v7.6.2/x-pack/filebeat/processors/decode_cef/decode_cef.go#L99 - https://github.com/menderesk/beats/blob/v7.6.2/x-pack/filebeat/processors/decode_cef/decode_cef.go#L99 - https://github.com/mschneider82/fx/blob/v1.9.0/app_test.go#L469 - https://github.com/neilotoole/sq/blob/v0.15.4/libsq/core/errz/errz.go#L42 - https://github.com/neilotoole/sq/blob/v0.15.4/libsq/core/errz/errz.go#L43 - https://github.com/nexmoinc/gosrvlib/blob/v1.37.5/pkg/validator/custom_test.go#L171 - https://github.com/nexmoinc/gosrvlib/blob/v1.37.5/pkg/validator/validator_test.go#L170 - https://github.com/ns1/pulsar-routemap/blob/v0.0.3/internal/validate/output.go#L41 - https://github.com/ns1/pulsar-routemap/blob/v0.0.3/pkg/validator/validator.go#L120 - https://github.com/ns1/pulsar-routemap/blob/v0.0.3/pkg/validator/validator.go#L220 - https://github.com/o4eredko/configuro/blob/v0.0.5/error.go#L56 - https://github.com/o4eredko/configuro/blob/v0.0.5/configuro_test.go#L799 - https://github.com/o4eredko/configuro/blob/v0.0.5/configuro_test.go#L855 - https://github.com/o4eredko/configuro/blob/v0.0.5/configuro_test.go#L907 - https://github.com/o4eredko/configuro/blob/v0.0.5/validate.go#L32 - https://github.com/odahu/odahu-flow/packages/operator/blob/58c3220a266a/pkg/apiserver/routes/v1/packaging/model_packaging_validation_test.go#L580 - https://github.com/odahu/odahu-flow/packages/operator/blob/58c3220a266a/pkg/apiserver/routes/v1/training/model_training_validation_test.go#L643 - https://github.com/odahu/odahu-flow/packages/operator/blob/58c3220a266a/pkg/apiserver/routes/v1/training/model_training_validation_test.go#L667 - https://github.com/odahu/odahu-flow/packages/operator/blob/58c3220a266a/pkg/apiserver/routes/v1/deployment/model_deployment_validation_test.go#L311 - https://github.com/odahu/odahu-flow/packages/operator/blob/58c3220a266a/pkg/service/connection/connection_service.go#L91 - https://github.com/odahu/odahu-flow/packages/operator/blob/58c3220a266a/pkg/service/connection/connection_service.go#L115 - https://github.com/odahu/odahu-flow/packages/operator/blob/58c3220a266a/pkg/service/batch/service_validate.go#L94 - https://github.com/open-telemetry/opentelemetry-collector-contrib/extension/observer/ecsobserver/blob/v0.55.0/docker_label_test.go#L161 - https://github.com/open-telemetry/opentelemetry-collector-contrib/extension/observer/ecsobserver/blob/v0.55.0/error.go#L49 - https://github.com/open-telemetry/opentelemetry-collector-contrib/extension/observer/ecsobserver/blob/v0.55.0/error.go#L73 - https://github.com/open-telemetry/opentelemetry-collector-contrib/extension/observer/ecsobserver/blob/v0.55.0/filter_test.go#L200 - https://github.com/open-telemetry/opentelemetry-collector-contrib/extension/observer/ecsobserver/blob/v0.55.0/exporter_test.go#L129 - https://github.com/open-telemetry/opentelemetry-collector-contrib/extension/observer/ecsobserver/blob/v0.55.0/exporter_test.go#L242 - https://github.com/open-telemetry/opentelemetry-collector-contrib/receiver/activedirectorydsreceiver/blob/v0.55.0/scraper.go#L258 - https://github.com/origoss/beats/blob/v7.5.2/x-pack/filebeat/processors/decode_cef/decode_cef.go#L99 - https://github.com/orion7r/br/blob/v5.1.0/pkg/restore/import.go#L331 - https://github.com/orion7r/br/blob/v5.1.0/pkg/restore/backoff_test.go#L75 - https://github.com/orion7r/br/blob/v5.1.0/pkg/restore/backoff_test.go#L92 - https://github.com/orion7r/br/blob/v5.1.0/pkg/restore/backoff_test.go#L105 - https://github.com/orion7r/br/blob/v5.1.0/pkg/restore/systable_restore.go#L103 - https://github.com/pingcap/br/blob/v5.1.4/pkg/utils/backoff_test.go#L74 - https://github.com/pingcap/br/blob/v5.1.4/pkg/utils/backoff_test.go#L91 - https://github.com/pingcap/br/blob/v5.1.4/pkg/utils/backoff_test.go#L104 - https://github.com/pingcap/br/blob/v5.1.4/pkg/restore/import.go#L341 - https://github.com/pingcap/br/blob/v5.1.4/pkg/restore/systable_restore.go#L103 - https://github.com/pingcap/tidb/blob/a2fe74fc92ed/dumpling/export/sql.go#L73 - https://github.com/pingcap/tidb/blob/a2fe74fc92ed/br/pkg/utils/backoff_test.go#L61 - https://github.com/pingcap/tidb/blob/a2fe74fc92ed/br/pkg/utils/backoff_test.go#L73 - https://github.com/pingcap/tidb/blob/a2fe74fc92ed/br/pkg/utils/backoff_test.go#L95 - https://github.com/pingcap/tidb/blob/a2fe74fc92ed/br/pkg/utils/backoff_test.go#L124 - https://github.com/pingcap/tidb/blob/a2fe74fc92ed/br/pkg/utils/backoff_test.go#L160 - https://github.com/pingcap/tidb/blob/a2fe74fc92ed/br/pkg/restore/import.go#L346 - https://github.com/pingcap/tidb/blob/a2fe74fc92ed/br/pkg/restore/import.go#L456 - https://github.com/pingcap/tidb/blob/a2fe74fc92ed/br/pkg/restore/systable_restore.go#L103 - https://github.com/projectdiscovery/notify/blob/v1.0.2/pkg/providers/providers.go#L114 - https://github.com/recall704/br/blob/v5.0.1/pkg/restore/import.go#L254 - https://github.com/recall704/br/blob/v5.0.1/pkg/restore/backoff_test.go#L75 - https://github.com/recall704/br/blob/v5.0.1/pkg/restore/backoff_test.go#L92 - https://github.com/recall704/br/blob/v5.0.1/pkg/restore/backoff_test.go#L105 - https://github.com/richard-mauri/beats/blob/v7.6.2/x-pack/filebeat/processors/decode_cef/decode_cef.go#L99 - https://github.com/romdo/go-validate/blob/v0.1.1/error.go#L66 - https://github.com/romdo/go-validate/blob/v0.1.1/validator.go#L77 - https://github.com/romdo/go-validate/blob/v0.1.1/error_test.go#L260 - https://github.com/romdo/go-validate/blob/v0.1.1/error_test.go#L329 - https://github.com/romdo/go-validate/blob/v0.1.1/error_test.go#L403 - https://github.com/saturn4er/fx-inject/blob/v1.4.0/app_test.go#L238 - https://github.com/sdzyba/yab/blob/v0.21.1/plugin/plugin_test.go#L65 - https://github.com/sdzyba/yab/blob/v0.21.1/plugin/plugin_test.go#L81 - https://github.com/selonsy/beats/v7/blob/v7.13.6/x-pack/filebeat/processors/decode_cef/decode_cef.go#L96 - https://github.com/sherifabdlnaby/configuro/blob/v0.0.2/error.go#L56 - https://github.com/sherifabdlnaby/configuro/blob/v0.0.2/configuro_test.go#L779 - https://github.com/sherifabdlnaby/configuro/blob/v0.0.2/configuro_test.go#L835 - https://github.com/sherifabdlnaby/configuro/blob/v0.0.2/configuro_test.go#L887 - https://github.com/sherifabdlnaby/configuro/blob/v0.0.2/validate.go#L32 - https://github.com/shirchen/fx/blob/v1.12.0/app_test.go#L531 - https://github.com/sidleal/beats/blob/v7.4.2/x-pack/filebeat/processors/decode_cef/decode_cef.go#L99 - https://github.com/snappyflow/beats/v7/blob/v7.0.24/x-pack/filebeat/processors/decode_cef/decode_cef.go#L96 - https://github.com/strangelove-ventures/ibctest/blob/80fc355a3e99/ibc/tx_test.go#L25 - https://github.com/strangelove-ventures/ibctest/blob/80fc355a3e99/ibc/packet_test.go#L40 - https://github.com/subratohld/modules/db/sql/mariadb/blob/fc3290068762/db.go#L119 - https://github.com/subratohld/sqldb/blob/73b7fe630a15/db.go#L114 - https://github.com/sudeepknair/beats/blob/v7.5.0/x-pack/filebeat/processors/decode_cef/decode_cef.go#L99 - https://github.com/tencentblueking/beats/blob/v7.6.2/x-pack/filebeat/processors/decode_cef/decode_cef.go#L99 - https://github.com/tikv/migration/br/blob/47f66b7edcf1/pkg/utils/backoff_test.go#L61 - https://github.com/tikv/migration/br/blob/47f66b7edcf1/pkg/utils/backoff_test.go#L73 - https://github.com/tikv/migration/br/blob/47f66b7edcf1/pkg/utils/backoff_test.go#L95 - https://github.com/tikv/migration/br/blob/47f66b7edcf1/pkg/utils/backoff_test.go#L124 - https://github.com/tikv/migration/br/blob/47f66b7edcf1/pkg/utils/backoff_test.go#L158 - https://github.com/tikv/migration/br/blob/47f66b7edcf1/pkg/restore/import.go#L346 - https://github.com/turbulent376/kit/blob/v0.0.2/config/configuro/error.go#L56 - https://github.com/turbulent376/kit/blob/v0.0.2/config/configuro/configuro_test.go#L946 - https://github.com/turbulent376/kit/blob/v0.0.2/config/configuro/configuro_test.go#L1002 - https://github.com/turbulent376/kit/blob/v0.0.2/config/configuro/configuro_test.go#L1054 - https://github.com/turbulent376/kit/blob/v0.0.2/config/configuro/validate.go#L32 - https://github.com/twpayne/chezmoi/v2/blob/v2.18.1/internal/cmds/lint-whitespace/main.go#L91 - https://github.com/twpayne/chezmoi/blob/v1.8.11/internal/cmd/lint-whitespace/main.go#L98 - https://github.com/vectornetworkteam/notive/blob/46cfdbf4a003/pkg/providers/providers.go#L114 - https://github.com/warsky007/beats/blob/v7.6.2/x-pack/filebeat/processors/decode_cef/decode_cef.go#L99 - https://github.com/webiks/beats/blob/v7.5.2/x-pack/filebeat/processors/decode_cef/decode_cef.go#L99 - https://github.com/xmidt-org/arrange/blob/v0.3.0/invoke_test.go#L115 - https://github.com/xmidt-org/themis/blob/v0.4.8/token/transport.go#L56 - https://github.com/xmidt-org/themis/blob/v0.4.8/token/transport_test.go#L434 - https://github.com/xmidt-org/themis/blob/v0.4.8/token/transport_test.go#L592 - https://github.com/xqk/ox/blob/v0.1.0/pkg/util/ogo/serial_test.go#L40 - https://github.com/xuhuaiyu/br/blob/v5.1.2/pkg/restore/import.go#L331 - https://github.com/xuhuaiyu/br/blob/v5.1.2/pkg/restore/backoff_test.go#L75 - https://github.com/xuhuaiyu/br/blob/v5.1.2/pkg/restore/backoff_test.go#L92 - https://github.com/xuhuaiyu/br/blob/v5.1.2/pkg/restore/backoff_test.go#L105 - https://github.com/xuhuaiyu/br/blob/v5.1.2/pkg/restore/systable_restore.go#L103 - https://github.com/yarpc/yab/blob/v1.19.1/plugin/plugin_test.go#L65 - https://github.com/yarpc/yab/blob/v1.19.1/plugin/plugin_test.go#L81 - https://github.com/zcong1993/notifiers/v2/blob/v2.1.0/combine.go#L32 - https://github.com/zcong1993/notifiers/v2/blob/v2.1.0/combine.go#L56 - https://github.com/zpatrick/testx/blob/v0.0.1/suite/suite.go#L85 - https://github.com/ptxmac/multierrgroup/blob/v0.0.1/multierrgroup_test.go#L24 - https://github.com/ptxmac/multierrgroup/blob/v0.0.1/README.md#L51 - https://github.com/ptxmac/multierrgroup/blob/v0.0.1/samples/thesolution/main.go#L27 - https://github.com/ptxmac/multierrgroup/blob/v0.0.1/samples/theproblem/main.go#L27 - https://github.com/uber-go/fx/blob/v1.9.0/app_test.go#L469 - https://github.com/uber-go/multierr/blob/v1.8.0/error.go#L41 - https://github.com/uber-go/multierr/blob/v1.8.0/error.go#L189 - https://github.com/uber-go/multierr/blob/v1.8.0/example_test.go#L65 - https://github.com/viamrobotics/goutils/blob/f21845865590/error.go#L24 - https://github.com/knative-sandbox/eventing-kafka/blob/v0.33.0/pkg/common/commands/resetoffset/controller/offsets.go#L264 - https://github.com/knative-sandbox/eventing-kafka/blob/v0.33.0/pkg/common/commands/resetoffset/controller/dataplane_test.go#L264 - https://github.com/pathwar/pathwar/blob/v2.78.0/go/pkg/pwagent/daemon.go#L89

[josharian]

182/5981 = ~3%

I clicked haphazardly through the list. I ignored instances that were part of multierr implementations, which was a fair few of these. I saw three major clusters in what remained:

1. Log each error individually. I'm a bit puzzled by this, insofar as it means that you lose the context that all these errors occurred in a single unit. And because I'm puzzled, I can't tell whether people would rather go exactly one level deep (why would you?) or would rather log all leaf errors (you'd lose structure, but then again you already are). I kind of suspect that this may primarily reflect difficulties in formatting multi-errors, rather than a specific desire to extract a one-level-deep slice.
2. Check whether a predicate applies to any element of the slice. This seems best served by some kind of Walk.
3. Write a test assertion that a particular set of errors occurred. I don't know whether testing error tree equality is in scope.

My take-away is that there isn't much of a case for extracting an []error exactly one level deep...but I didn't believe that in the first place, so there could definitely be confirmation bias here. :) Probably worth doing your own analysis too.

[BenjamenMeyer]

My take-away is that there isn't much of a case for extracting an []error exactly one level deep...but I didn't believe that in the first place, so there could definitely be confirmation bias here. :) Probably worth doing your own analysis too.

Since the current built-in Unwrap only goes one-level deep by default, preserving it that way would minimize code breakage while having Split() []error to get the full tree would allow the additional use-cases for those that want it for tree walking or anything else.

1. Log each error individually. I'm a bit puzzled by this, insofar as it means that you lose the context that all these errors occurred in a single unit. And because I'm puzzled, I can't tell whether people would rather go exactly one level deep (why would you?) or would rather log all leaf errors (you'd lose structure, but then again you already are). I kind of suspect that this may primarily reflect difficulties in formatting multi-errors, rather than a specific desire to extract a one-level-deep slice.

In a previous project I worked on folks generated their own multi-error and dumped each out individually to minimize log space - but it meant you had a bunch of logs lines that made up the entire single log message. This would probably be another good use of tree walking.

In a current project I see folks comparing to a string b/c you can't by default go more than one level deep (see my example from https://github.com/golang/go/issues/53435#issuecomment-1168913889) and yes it's super fragile (especially if one tries to add translations into the mix) but it's due to current limits. Fixing the multi-error functionality would probably help most of these eventually go away as folks update the projects and learn about the functionality.

3. Write a test assertion that a particular set of errors occurred. I don't know whether testing error tree equality is in scope.

If errors.Is() and errors.As() could detect the individual errors desired then it shouldn't be necessary to test for tree equality as part of this effort. If desired it could be addressed in the future, but the implementation would be simple enough - perhaps best left for when two slices could be easily compared via a generic.

[neild]

I agree that the data seems to indicate that unwrapping one level of []error is rare, and that in the rare cases that it is done the justification is often dubious.

Looking back over the above discussion, I think there are three major questions.

1. Should we provide Join?

   @jimmyfrasche says:

   I see the utility of including a basic multierror in std, but just gluing the errors together with a sep seems like it could get messy when multierrors get composed. I think, like a tree walking API, it should be left out and experimentation allowed to continue outside std for now.

   Looking at some existing multierr implementations:

   • github.com/hashicorp/go-multierror formats as a * bulleted list with a header line.
   • go.uber.org/multierr joins the errors with ;, and formats %+v as a * bulleted list with a header line.
   • tailscale.com/util/multierr formats as a - bulleted list with a header line.

   https://go.dev/play/p/zwGWvtR0IHW

   It seems as if existing implementations have converged on providing a list with one error per line, as an option if nothing else. This may argue that Join is the wrong API here.

2. Should we provide Split?

   @josharian says:

   So far, I see no evidence that anyone cares much about the structure of the error tree. It looks to me like they are all: misusing the API (evidence that the API is wrong); using the wrong API (evidence that the API is wrong); using the API pointlessly; or using the API to evaluate a predicate "anywhere along the error chain".

   I think that the evidence from calls to "go.uber.org/multierr".Errors supports this argument. It feels very strange to me to not provide a simple way to unpack a multierr, but I'm convinced that this API is an invitation to misuse. The rare cases that actually need to unpack one level of a multierr can use a type assertion and call the Errors method directly.

3. Should we provide a tree walking API?

   None of the existing multierr implementations I've looked at provide this. If this is useful, we should demonstrate it outside the standard library first.

[timothy-king]

One additional pattern I saw for Errors() was as a way to append additional errors to an existing list of errors. Essentially re-flattening the list. Maybe not worrying about the distinction of Join(";", Join(";", i, j), k) and Join(";", i, j, k) simplifies something else? Not really sure why though.

[timothy-king]

FWIW A clever use of Join can cover some of the headerline cases Join("\n *", errors.New(headerline), errs...). This will compose terribly with trees though. Maybe we need a format for Join that is tree friendly?

[josharian]

Maybe not worrying about the distinction of Join(";", Join(";", i, j), k) and Join(";", i, j, k) simplifies something else?

That's how the Tailscale multierr package works. The reasoning is that it lets you write code that builds it incrementally:

var me Multierr
// ...
me = Multierr(me, err)
// ...
me = Multierr(me, err)
// ...
return me

In this case case, the final return will return nil, a single error, or a multierr composed purely of the non-nil errors encountered along the way.

I almost wrote this earlier. But I didn't, when I realized that this is effectively equivalent to:

var errs []error
// ...
errs = append(errs, err)
// ...
errs = append(errs, err)
// ...
return errors.Join(errs)

The only difference is that, as originally proposed, Join doesn't return the sole non-nil error directly in the case in which there is exactly one non-nil error. But that could be changed. :)

[josharian]

FWIW A clever use of Join can cover some of the headerline cases

I think for that you want: fmt.Errorf("THE HEADER: %w", errors.Join(errs)).

[earthboundkid]

This will compose terribly with trees though. Maybe we need a format for Join that is tree friendly?

I think this is the case for errors.Walk: most of the time you're going to be logging into some sort of structured reporting system as opposed to just outputting text to stdout, so for that you'll want the tree displayed as nested HTML tags or whatever. I think that if errors.Join just always used semicolons with %v and new lines plus bullets with %+v, that would get you a lot of the way towards formatting the stdout text correctly, and for any other more complicated case, it could be handled by Walk.

[josharian]

Should we provide Join?

If we don't, there is no standard way to go from an unknown number of errors to a single error (or nil). fmt.Errorf supporting multiple %w verbs is awesome, but only supports a fixed number of errors.

I think we need some API that goes from an unknown number of errors to a single error, because this is a very common use case. It could be one-shot (take in a slice) or incremental (add errors one at a time).

I don't have strong feelings about exactly what that API is. I was happy with Join and would probably be happy with others as well.

[neild]

I very much like the simplicity of a join function that takes a []error and returns an error, as opposed to an append function which adds more errors to an existing one. I think join is the right API for the standard library. Append opens too many questions about tree structure and allocations.

The sticky question is what the text of a joined error should be. The nice thing about Join(sep, errs) with an explicit separator is that it has no opinion; the user remains under full control of the error text. The problem with it is that existing implementations have a clear preference for bulleted lists of errors, which you can't do with just a separator. So perhaps we can't get away from the need to be somewhat opinionated about error text.

[BenjamenMeyer]

If we don't, there is no standard way to go from an unknown number of errors to a single error (or nil). fmt.Errorf supporting multiple %w verbs is awesome, but only supports a fixed number of errors.

How about expanding %w to something like:

• %Nw - Nth error in the list, or nil if the list isn't that long, following the pattern of %Nd formatting of numbers
• %w - single, top most error, equivalent of %0w
• %#w - all errors (though of %*w but since #v is already for verbosity, why not use it with %w too?)

Then you could:

fmt.Errorf("%w: had some error due to %1w", err, err)
fmt.Errorf("Had errors: %#w", err)

I very much like the simplicity of a join function that takes a []error and returns an error, as opposed to an append function which adds more errors to an existing one. I think join is the right API for the standard library. Append opens too many questions about tree structure and allocations.

The sticky question is what the text of a joined error should be. The nice thing about Join(sep, errs) with an explicit separator is that it has no opinion; the user remains under full control of the error text. The problem with it is that existing implementations have a clear preference for bulleted lists of errors, which you can't do with just a separator. So perhaps we can't get away from the need to be somewhat opinionated about error text.

func (err error) Join(sep string, errs []error) error

Then sep could be -, \n\t-, etc as opposed to just making sep a single rune.

$0.02 FWIW

[jimmyfrasche]

My main argument against Join/Split is that it's easy to punt that decision to 3rd party implementations to work out the details before anything gets enshrined in the Go 1 compatibility agreement. Defining the interface and how Is/As work is enough to let everything work together and experiment.

[neild]

Third party implementations have been around for a while. Is there anything more that's going to be worked out in them so far as error text goes? Or to put it another way, what new information do we think we're going to get?

Options I see:

1. Drop Join.

   I don't like this option. I think there's compelling evidence that Split isn't necessary, but combining a list of errors into a single error is not just a common use case, it's fundamental to every current use of multierrs.

2. Join with a user-provided separator, as in my original proposal. Simple and unopinionated, but out of step with common practice of formatting multierrs as a bulleted list. You can still join with "\n" for one error per line, which might be good enough.

3. Join with an opinionated separator. The Error text separates errors with ; and %+v formats as a multi-line bulleted list.

   I worry that this might conflict with using %+v to add stack information to errors in the future. (Proposed for 1.13; subsequently withdrawn, but perhaps we'll want to take another stab at it someday.)

4. Join with an opinionated separator, formatting as a multi-line bulleted list by default.

   The argument in favor of this approach is that two out of three third-party implementations surveyed already do this, and glomming many errors into a single line is gets quite unreadable very quickly. I think I'm coming around to this being the right approach.

[timothy-king]

I don't have a strong opinion about the difference between 3 and 4. For bullet lists, I think the package would need to have an opinion about trees. For trees, adding an indentation per list layer (including length 0 and 1) seem reasonable. That or commit to one level of bullets for the left to right visit order of the leaves.

One way we could make String() being opinionated less risky is to have a clear alternative if you want a different format. Would wrapping in a custom error type if you want an alternative format be easy enough?

[josharian]

Musing...one downside to the multi-line w/ bullets formatting is that it doesn't nest well (without magic).

Consider errors.Join(errors.Join(errors.New("A"), errors.New("B")), errors.Join(errors.New("C"), errors.New("D"))). How should this format using e.g. "newlines+asterisk"? IIUC, it'd naively format as:

* * A
* B
* * C
* D

[jimmyfrasche]

If it did have magic it would be easily defeatable by wrapping the inner list before adding it to the outer list.

[neild]

Sampling of existing packages handling formatting of nested multierrs: https://go.dev/play/p/7L2Ydp4-JdZ

[josharian]

Uber appears to all have some magic going on(?). Note that adding a third level changes the output significantly: https://go.dev/play/p/yU8I9Hiswzq. Another test case: https://go.dev/play/p/Ms0YULlXaon.

[neild]

I think Uber's behavior is because it doesn't plumb the %+v down into contained errors, so everything under the top level formats as ;-separated on a single line.

[josharian]

Looking at that dog's breakfast of outputs makes me think that the original Join proposal is the best of the lot. But I really don't feel strongly.

[dsnet]

Several thoughts:

1. We should not expose the ability to specify the delimiter to Join.

   • Rather, we should choose something reasonable, whether it be ; or \n or even switching between the two based on whether the error is formatted with %v or %+v.

   • Allowing users to choose is just going to lead to more mess. One package uses ;, another uses \n, yet another ,. When I compose all of those errors together, the result is a big mess.

2. A potential use-case for decomposing a multi-error is with non-fatal JSON (or protobuf) deserialization.

   • In such an application there could be a number of non-fatal errors where the user wants to have as much of the unmarshal process to proceed, but to remember every non-fatal error (e.g., mismatching types, invalid UTF-8, etc.). Afterwards, the caller inspects the set of errors to do additional work (e.g., extract the errors to present it more nicely in a UI, ignore some of them, do extra cleanup for some, etc.).

   • This doesn't need to be solved using Split, but we definitely need a way to iterate through the errors in a multi-error. The JSON package could document that it is guaranteed to return a flattened list of errors such that a single call to errors.Split would return everything.

3. A gotcha with multi-errors is that is that errors.Is is ill-defined as whether it means exactly is or contains. The former checks that every element in the multi-error is the target error. The latter checks whether any element in the multi-error is the target error. It's a subtle difference, but has quite a significant impact.

   • For example, there used to be a multi-error package in google.golang.org/protobuf to hold non-fatal errors (e.g., invalid UTF-8, unpopulated required fields, etc.).

   • Suppose someone wanted to ignore required-not-set errors, it would be tempting do something like:

     switch err := proto.Unmarshal(b, &m); {
     case errors.Is(err, proto.ErrRequiredNotSet):
         // ignore required fields that are unpopulated
     case err != nil:
         return err
     }
     ... // use m

     the intent of the user is to ignore only required-not-set errors, but to still reject other errors.

   • However, the seemingly innocent logic above is buggy if the error returned by proto.Unmarshal was errors.Join(proto.ErrInvalidUTF8, proto.ErrRequiredNotSet) as it would subtly ignore the presence of the proto.ErrInvalidUTF8 error.

   • Regarding this issue, I don't have any good solutions. There are times when I would want errors.Is to mean "exactly is" and other times that I want it to mean "contains". Either semantic we choose, it's going to be footgun.

   • We avoided this problem by just not depending on multi-errors and to instead depend on options (e.g., proto.UnmarshalOptions.AllowPartial) to prevent the generation of proto.ErrRequiredNotSet in the first place. This is one way to work around the issue, but I still think it would have been nice for proto.Unmarshal to return a multi-error (e.g., a list of all errors annotated by the path and the error that occurred).

[BenjamenMeyer]

• Regarding this issue, I don't have any good solutions. There are times when I would want errors.Is to mean "exactly is" and other times that I want it to mean "contains". Either semantic we choose, it's going to be footgun.

Perhaps we need a new errors.Has or errors.Contains to allow for the contains version and then specify that errors.Is is the exact error. More choice of methods may be the solution.

[neild]

Every multierr implementation I've surveyed has defined errors.Is(multierr, target) as true if target matches any error in multierr. There may be a use case for an exact match, but I haven't observed it in the wild. I think this argues for this being the generally expected behavior.

[neild]

Looking at that dog's breakfast of outputs makes me think that the original Join proposal is the best of the lot

I think the evidence here is that a bulleted list is not the right formatting option for the Error method. None of the implementations surveyed do a good job at all of formatting nested multierrs. (Uber comes the closest, by virtue of only applying the bulleted-list formatting to the outermost error.)

That leaves a user-provided delimiter (the original Join proposal) or a fixed delimiter.

How about "\n"? Semicolon-separated errors get unreadable quickly, and placing each error on a separate line gets most of the benefits of a bulleted list without the problems caused by nested indenting. Tree structure of nested errors gets lost, but

// Join returns an error that wraps the given errors.
// Any nil error values are discarded.
// The error formats as the text of the given errors, separated by newlines.
// Join returns nil if errs contains no non-nil values.

func Join(errs ...error) error

@josharian's last two test cases of nested errors using newlines as a separator:

https://go.dev/play/p/a7gHGrNst9- https://go.dev/play/p/c1Y7GYwEAyj

[josharian]

This has a been a very long conversation, but it has also been focused and productive and fun, and I think we're starting to converge(?). Thank you.

One small tweak I would suggest to the Join docs:

// If errs contains exactly one non-nil error, Join returns that error.

This avoids pointless wrapping.

...and I just realized I mixed two topics in a single thread, which makes emoji voting hard. Oops. I'll assume any thumbs up/down are purely for the technical content. :)

[BenjamenMeyer]

How about \"\n\

The issue with \n is programs needing to parse the log output being able to associate the lines together.

[josharian]

Programs parsing raw line-oriented log output are already going to have a bad time on any number of fronts. And there are proper fixes (structured output) and cheap hacks (a passthrough io.Writer that rewrites non-trailing \n to \n\t, or just using a standard logging prefix) to handle that.

Ironically, given that many people choose to log multi-errors individually, plain \n might make some systems easier.