Closed jabolopes closed 2 years ago
Error-agnostic generic APIs
I think it's wroth pointing out, that this doesn't need an stdlib ErrorOr
type to happen. You can write those APIs and let the user decide to just use struct{ T; error }
directly, write their own ErrorOr
type, or use one from a third party library. A package to do that only needs a handful of lines of code and would allow the community to vote with their feet.
Errors / values in channels, maps, slices
Again, there is no need for this to live in the stdlib. Note that even today, it's not uncommon to just define a custom type Data struct { A T1; B T2 }
, if you need to send multiple values over a channel. And struct { Val T; Err error }
doesn't seem particularly special here - or above, for that matter.
There are APIs that are exception to 4 states of
(T, error)
rule, such as, theio.Reader.Read
.
I think an a little bit more instructive example is io.Writer.Write
. Short writes are normal, need to be signalled somehow and Write
(as opposed to Read
) can't be safely retried to get the error on a second call.
You treat this "exception" as bad, say that ErrorOr
would allow us to get rid of it. But we could have already decide just not to have it - we could've just specified that Read
and Write
must not return an error with a non-zero number of bytes read. The reasons we didn't have nothing to do with the absence of ErrorOr
, really. It's that these APIs need to be able to return both.
But really, that's besides the point. Because I think if anything ErrorOr
should be tried out via third party libraries first anyways, which will, either way, leave us with APIs doing both (T, error)
and ErrorOr
, so APIs which need it (and developers who want it) can just use the existing mechanism anyways.
This is out of scope for this proposal but I think it's an interesting, possible future extension of the
ErrorOr
idea, that can help with Go exceptions without actually requiring exceptions.
I think what you describe here is essentially the try
proposal, which a) was retracted because people didn't like it and b) shows that this doesn't really need ErrorOr
to work - it works just as well with the existing multiple return values.
This doesn't look like it enables any shorter, safer, or clearer form of error handling, and instead just adds an extra layer of indirection. If done in the standard library, this would be a significant churn of what is considered idiomatic for what is arguably little benefit, and it can't even handle the common usecase of multiple return values, resulting in even more inconsistency.
@merovius
I think it's worth pointing out, that this doesn't need an stdlib
I agree. It doesn't need to be in the stdlib. The only reason I mentioned the stdlib is because there might be error-returning functions in the stdlib that would like to use ErrorOr
. In this case, I doubt very much that the stdlib maintainers would introduce a dependency on a third party package. But running this as a trial from a third-party package or in the experiments (exp
) package would also be a good approach.
It's that these APIs need to be able to return both.
I agree with you but I think a little more needs to be said. The io.Reader.Read
documentation says:
"""(...) a Reader returning a non-zero number of bytes at the end of the input stream may return either err == EOF or err == nil."""
So, if we consider the following case:
Read
returns some bytes, nilRead
returns 0, io.EOFThis is equivalent to the ErrorOr[int]
case. So the behaviour I proposed is already in Read
, but it just happens to be one of the many cases that Read
returns. So I was proposing narrowing it down to this one case.
But at the same time, if I think really long term, I imagine that at some point the ErrorOr
could become the standard way of reporting errors in Go. This means that the majority of APIs would be using ErrorOr
and only a minority of APIs (like io.Writer.Write
) would be using (T, error)
.
At this point, the (T, error)
would no longer be mistaken with ErrorOr[T]
semantics because it would only be used by functions that actually need to return 2 values (like you said) since all the other functions that either return a value or an error would be using ErrorOr
. At that point, I wouldn't want to change interfaces like io.Writer.Write
at all, and perhaps even io.Reader.Read
, because the ambiguity is gone.
try proposal (...) people didn't like it
Can you point me to the feedback for the try proposal so that I can evaluate whether the feedback applies to the ErrorOr
proposal or not? Thanks!
@seankhliao
This doesn't look like it enables any shorter, safer, or clearer
The following is shorter (less chars), safer and clearer (it's either error or value, no ambiguity):
func (...) Error[int] {
than:
func (...) (int, error) {
This is shorter and clearer (less variables):
x := myfunc1()
than:
x, err := myfunc1()
This is shorter (less args / return values) and clearer (API type shows it's error-agnostic):
func (g *Group[K, V]) Do(key K, fn func()V) (v V, shared bool)
than:
func (g *Group[K, V]) Do(key Key, fn func()(V, error)) (v V, err error, shared bool)
This is shorter and clearer:
chan ErrorOr[int]
than:
type IntOrError {
v int
err error
}
chan IntOrError
idiomatic
The question of idiomacy is very debatable because the current error idiom was established in a pre-generics era. Now, with generics we want to rethink what the idiom should look like. That's the whole point of issues like this one and also https://github.com/golang/go/discussions/48287.
If we don't allow the idiom to change ever, then we will never be able to use generics to their full potential. The whole point of a feature like generics is so that we can improve the language and its libraries across and reach a new level of idiom.
it can't even handle the common usecase of multiple return values
I think it can, e.g., ErrorOr[struct{int; float32}]
. And that's just one possibility. There are many others.
Also, if Go ever supports tuples, we can also combine ErrorOr
with tuples, e.g., ErrorOr[(int, int)]
, (assuming (x, y) is the tuple syntax).
The error handling proposals have been focused on checking the error part, not the declarations. Saving a few characters on the function declaration is missing the point.
x := myfunc1()
is arguably less clear, as without context (such as knowing the type), it doesn't look like there is an error path in here.
I'm not saying that idioms can't change, but they have to bring benefits proportional to the cost they incur. Right now the benefits of this proposal aren't very clear.
Types are written once, but the values they produce are used many times.
x := foo()
if x.Err() != nil {
return x.Err()
}
vs
x, err := foo()
if err != nil {
return err
}
Here, it's longer and harder to read, with more symbols in the way.
ErrorOr[struct{int; float32}]
Feels like a step backwards, your type declaration is now much longer (and you haven't named the fields yet), and if you move the struct to be a named type, you now have to deal with the an inflation of types that exist solely to return multiple values when the language can already do that. You're also forcing return values to be grouped together when they may have no reason to be.
A somewhat minor point, but I think of func F() (int, error)
as a function that returns an int
(and also an error
). If I write func F() ErrorOr[int]
then it looks like I have a function that returns an error
(and also an int
). For a function that returns a value, the fact that it also returns an error is not an important fact about that function. It would be nicer if it didn't take precedence when writing the function signature.
@jabolopes
Can you point me to the feedback for the try proposal so that I can evaluate whether the feedback applies to the
ErrorOr
proposal or not? Thanks!
I did. There's probably more, on golang-nuts or the subreddit or the gopher slack, but TBQH I'm not really better at finding that than you and the github issue should be plenty.
IMO it is pretty much out of the question that the same arguments apply. The only real difference between the two is that one spells it try(F())
, while the other spells it <-F()
. Apart from that, they seem to be exactly equivalent. Which was the real point: Doing anything like that doesn't require ErrorOr
.
I agree. It doesn't need to be in the stdlib. The only reason I mentioned the stdlib is because there might be error-returning functions in the stdlib that would like to use ErrorOr. In this case, I doubt very much that the stdlib maintainers would introduce a dependency on a third party package. But running this as a trial from a third-party package or in the experiments (exp) package would also be a good approach.
FWIW the true benefit of providing it as a third party package is that it makes this discussion completely obsolete. You could just write that package, publish it and publicize it and no one could object to that. If the community likes it, people will start using it. And instead of arguing about its value in the abstract, you could file a proposal in year or two pointing at all the projects using it, to make the case of moving it into the stdlib.
I highly doubt that an addition like this would make it into the stdlib without that path. Even if we would, say, add the type to errors
, 1. old stdlib function can't use it, because of backwards compatibility and 2. new functions wouldn't use it, before it has clearly crystallized as the accepted new idiom. Because consistency matters and while we can change conventions and live with the resulting inconsistency for a while, we wouldn't risk getting locked into a new convention which then gets deprecated again. That's why code which doesn't have to be in the stdlib, shouldn't start out there.
I think the history of github.com/pkg/errors
can serve as a good model here. It existed as an "unblessed" third party package for years, slowly gaining adoption by the community. At some point, it became clear that the community wants something like it enough to warrant trying to put something in the stdlib. So a proposal was filed, resulting in golang.org/x/exp/errors
, which eventually was moved into errors
. There are a couple extra steps here and github.com/pkg/errors
is not the only package which can claim heritage over this change. But the basic model is that this was done as a third party package and all the draft designs and discussions and proposals to move it into the stdlib happened after it was clear that the community wants something like it.
It just makes things better for everyone involved - you don't have to argue that it's a good idea and just do it, the Go projects can benefit from the experience gained by the community over the time as a third-party package and when we finally do add it, we can discuss bigger changes. Maybe even to the language proper. Maybe allowing the existing stdlib functions adopt it, without breaking compatibility.
@seankhliao
without context (...) it doesn't look like there is an error path in here.
The problem you are describing already exists in Go today and it relies on naming conventions (or context) for clarity. For example, the following examples are made clear not because we know the types of the variables but because we follow a naming convention. With ErrorOr
, it's no different.
x := f() // A value (i.e., not an error)
err := f() // An error
x, err := f() // A value and error
x, y := f() // 2 values
x, ok := f() // A value and a bool
x, y, ok := f() // 2 values and a bool
Types are written once, but the values they produce are used many times.
In the general case, we should assume that both the caller and callee use ErrorOr[T]
, so the example is like this:
x := foo()
if x.Err() != nil {
return x
}
and you haven't named the fields yet
A lot of APIs don't name the return values either when using (T, error)
. It's a matter of preference, not a strict requirement. With ErrorOr
, it should be no different.
grouped together when they may have no reason to be.
The reason is so that we can instantiate generic types such as T
and func()T
with errors ErrorOr[T]
and func()ErrorOr[T]
, respectively. This is not possible with the current error handling mechanism since (T, error)
cannot be used to instantiate T
and func()(T, error)
cannot be used to instantiate func()T
. This explained in detail in the section titled "Error-agnostic generic APIs".
@ianlancetaylor
It would be nicer if it didn't take precedence when writing the function signature.
I agree. If you have some ideas I'd love to hear more. Here are few ideas (in no particular order):
Change the name ErrorOr
to something that emphasizes the final value rather than the error. Naming is hard, but a few options could be Result[int]
or Checked[int]
or Safe[int]
or Handled[int]
, or any short form like R[int]
, C[int]
, S[int]
, etc. Suggestions are very much welcome!
Another way to go about this would be to use someting like Either[int, error]
, which is more similar to the (T, error)
approach. The disadvantage of this approach is that Either
does not name its methods as Val()
and Err()
but something more general like Left()
and Right()
so I think it wouldn't read as well on the calling side.
Another idea would be to have syntactic sugar for ErrorOr
, for example, writing int?
would mean ErrorOr[int]
. For example, func MyFunc() int?
would be syntactic sugar for func MyFunc() ErrorOr[int]
(assuming we're still using the ErrorOr
name, because that can also be changed). Instead of ?
we could also pick another syntax, e.g., int+
, int.
, int!
, etc, and even invert the order, e.g., ?int
, +int
, .int
, etc, which is probably even more consistent with the slice syntax, for which []
precedes the underlying type.
Does this go in the direction you're thinking? Feedback would be great!
@Merovius
I agree with your approach. I will create a third party package with the ErrorOr
so that community members can use it if they wish. I will create this package over the weekend and paste the link to the package back to this discussion.
A few more thoughts below:
The only real difference between the two
I don't want to lose sight of the bigger picture here. This proposal is to overcome the problems with the titled sections "The 4 states of (T, error)", the "Error-agnostic generic APIs", the "Errors / values in channels, maps, slices", the "Naming problem with err", the "Inconsistent use of (T, error)".
The section about "Future ideas" is out scope and is only meant as an idea. So I don't want to reduce the whole proposal to just the "Future ideas" section.
I will take a closer look at the try
proposal over the weekend to better evaluate the feedback.
discussion completely obsolete
The point of this discussion is to rethink error handling in the presence of generics. I think this got lost when my proposal's title was unilaterally changed to something with a different meaning. I have changed the title back and I'd prefer (if possible) that the title's meaning remain unchanged, although the actual wording can be changed.
This proposal is not about defining the ErrorOr
type, although getting feedback about the ErrorOr
type is also useful.
On the title, proposals should have a clear scope and design that can be evaluated and decided upon. If you want a general discussion of error handling possibilities, it's out of scope and should be done in one of the forums
@seankhliao
I followed @robpike 's suggestion to create a proposal and chose the title following his comment "You are asking for a new way to think about errors.". So I don't see what is wrong with that.
clear scope and design
This has a clear scope described in the "Objectives" section and a clear design described in the "Design ideas" section.
I would like to have a title along the lines of "thinking error handling in the presence of generics" without you unilaterally changing it without my consent. Please let me know how we can get there. I'm open to suggestions on wording / spelling if that's the blocking issue.
The section about "Future ideas" is out scope and is only meant as an idea. So I don't want to reduce the whole proposal to just the "Future ideas" section.
FWIW I did respond to all sections as well. Except, I guess, this one, where I also thought of something else to say:
The 4 states of
(T, error)
You are correct that there are 4 possible results. What I find significantly less clear is a) how much of a problem that is and b) how much of that problem your proposal solves.
There are three potential problems I can see with this:
To me, this proposal seems to try and solve things which appear non-problems to me, while doing little to address the things which I perceive as problems. I would like at least some sort of data on how much of a problem the things it's trying to address are.
This has a clear scope described in the "Objectives" section and a clear design described in the "Design ideas" section.
I would like to have a title along the lines of "thinking error handling in the presence of generics" without you unilaterally changing it without my consent. Please let me know how we can get there. I'm open to suggestions on wording / spelling if that's the blocking issue.
The scope, according to the "Objectives" section, is to introduce a new type. So, the title seems accurate. I agree with @seankhliao that a more general "rethinking error handling" is not an appropriate github issue or proposal and should be done in a place more conducive to open-ended discussion, like golang-nuts, reddit, slack, the twitter community or some other forum.
I also think that you can be expected to abide by the processes set out by the Go project. The people who use github issues the most to track their day-to-day work should also have the strongest word in how they are used. There is nothing "unilateral" about that - if anything, your want for a specific title incompatible with that usage is unilateral.
if we don't also address it statically
I'd be interested in hearing more about this. How would you go about addressing it statically?
The scope, according to the "Objectives" section, is to introduce a new type.
Yes, you're right. I updated the "Objectives" section to better reflect what I intended.
abide by the processes set out by the Go project
I want to abide by the processes and AFAIK I am abiding by the processes. This is why I said I can change the title. That's not a problem. I reviewed the proposals which doesn't say anything about the title of the Github issue, so AFAICT my title should be fine (unless I missed something). I also asked for the title policy to be shared with me, but I only got that link to the proposals, so I'm assuming that's all there is to it.
Unilateral in this case means that the title was changed by someone else without ever asking me to change the title, without giving me a chance to evaluate the policy (which I haven't seen still) and a choose a better title, without a explanation of why the title was changed, without a reasoning for the new title. That's what unilateral means.
I expect mods to use go through a process of first asking the proposer to change the title (also good is to propose new titles) and if the proposer is clearly not abiding by the processes, then the mods can use their permissions to change the title.
But that is not the case here. I have demonstrated willingness to work with the process and even shown openness to change the title. So this is clearly not the case of someone not abiding by the rules. So I don't think it's correct for the mods to change the title in this case because the problem has not escalated to a place that it requires action from the mods' side. I think that's a misuse of the mods' permissions.
Also, at this point, the context is already layed out in the discussion and it's clear that the title "ErrorOr type wrapping (T, error)" is wrong. So I think it's harmful to purposefully change the proposal's title back to a title that has already been discussed that is harmful for this proposal. I don't think this is right.
I would like to request once more that we can start a discussion to change the title to a new title that resembles more something like "rethinking error handling in the presence of generics". Are y'all willing to work with me to get us closer to that place? Or is this the end of the title discussion?
Also, at this point, the context is already layed out in the discussion and it's clear that the title "ErrorOr type wrapping (T, error)" is wrong.
I disagree. The name, as well as some details, might be up for debate, but ultimately, that's the concrete change you are proposing. And it's not at all uncommon to use placeholder names in proposal-titles, even if they are still up for debate. ErrorOr
is as good a place holder as Result
or anything else for that purpose.
You seem to instead want to make this a non-proposal and just be an open-ended discussion about how generics might influence error handling. But you've been told that this is not how github issues are used in the Go project and that such a discussion should happen on one of the forums more suited to that. So, if you want that, you might want to close this issue and start a discussion there.
Issue titles and labels are used by bots, searches and boards to triage and categorize issues, based on how they should be handled. These processes are not necessarily fully documented, relying instead on human judgement - generally, this is a good thing, as it allows more flexibility. Someone needs to make that judgement, and for that there is a set of people trusted to have enough of an understanding of the processes involved and triage issues accordingly. One such person has triaged this issue, to the best of their abilities. You might disagree with that triage, and there is some flexibility (e.g. if you prefer a different type name), but the basics of a) github issues are not for open-ended discussions, so the issue should focus on a concrete change and b) the title should reflect that change, won't budge.
The question is, is this really what you want to spend your time arguing about? Trust me when I say that, for the concrete change you are proposing, "introduce ErrorTo[T]
to wrap (T, error)
" is good enough for everyone involved to understand what the issue is about, even if it is not fully accurate. That's ultimately all that matters.
Based on the discussion above, this proposal seems like a likely decline. — rsc for the proposal review group
Hi everyone,
Following @robpike 's suggestion, I'd like to create a proposal for a new way to think about errors given that Go supports generics now.
This is similar in spirit the plan to think about APIs in the presence of generics but for error handling specifically.
Preliminary notes:
ErrorOr
,Val()
,Err()
, etc, are meant for illustration purposes only and any new names are welcome!Objective
Rethink Go's error handling mechanism in the presence of generics to overcome the following problems:
(T, error)
allows for 4 possible states but the majority of APIs only use 2 states and the other 2 states could be considered incorrect.T
and generic functions likefunc()T
cannot be instantiated with(T, error)
orfunc()(T, error)
respectively. This means API developers must artificially bake in errors in their APIs to allow callers to pass errors or pass error-returning functions.err
.(T, error)
The following are out of scope for this proposal:
context.Context
problem for APIs.(x, y, z, ...)
argument like a tuple.(T, error)
return like a tuple (see alternatives considered).Background
The 4 states of
(T, error)
The current mechanism to return errors is
(T, error)
which being a product type means it allows for 4 possible states, i.e., any combination of a proper / improper value, and an error ornil
. But the majority of APIs only care about either returning a value or an error, they don't care about the other 2 states, which could even be considered incorrect. Notable exceptions to this rule are theio.Reader.Read
API (more on this later).Because most APIs care only about 2 states, there is an opportunity to leverage the type system and generics to eliminate the 2 undesirable states, by introducing a new type
ErrorOr
that can only represent a value or an error, and doesn't allow for any other states. This would be used in new generics aware APIs.Error-agnostic generic APIs
One such API is the
singleflight.Do
, which could be reimagined with generics as:The
Do
API is error-agnostic, i.e., in principle it could accept infn
either an error-returning function or not because it doesn't do anything with the error. But in current generics, it's not possible to pass an error-returning function tofunc()V
sinceV
cannot be instantiated with multiple-return values or(T, error)
.This applies not only to
singleflight
but to any API that accepts a generic argumentT
or a generic functionfunc()T
, does not inspect that argument, but simply returns it later.To overcome this limitation, API developers must artificially bake in errors in their APIs, for example:
or:
both of which are suboptimal because even though this API is error-agnostic, this property was lost and it's not longer reflected in its type or enforced by the typesystem.
With
ErrorOr
, the burden of deciding on whether the API needs error handling or not goes away, and instead API callers have a free choice on whether to callDo
with an error-returning function (e.g.,func()ErrorOr[T]
) or not (e.g.,func()T
).Also, with
ErrorOr
the fact thatsingleflight.Do
is error-agnostic remains captured in the type and enforced by the typesystem. This is a property that is worth retaining.Another (small) benefit of the
ErrorOr
in this case is that thesingleflight.Group
implementation would be simpler because it would need to store only 1 value per call (i.e., theErrorOr[T]
) instead of storing 2 values per call (i.e., theT
and theerror
).Errors / values in channels, maps, slices
When I use channels to implement pipelines or glue computations together that can either return a value or fail with an error, I found myself having to define a new
struct
type to encapsulate the value / error to use in the channel, e.g.,chan IntOrError
(pre-generics).The
ErrorOr
would also be useful for channels, e.g,chan ErrorOr[int]
so we don't need to redefine new struct types for this use case likeValueOrError
(or a generic equivalent of that) because we can reuseErrorOr
instead.We could also store
ErrorOr[T]
in collections such as maps, slices, etc. For example, spawn 10 goroutines and have each store their result in[]ErrorOr[int]
. Another example, implement a cache with positive and negative caching, e.g.,map[K]ErrorOr[V]
.Naming problem with
err
When we use the pattern
(T, error)
, we need to define theerr
many times:In some cases, there are no new names on the left side of
:=
so we need a few tricks there, either by writingvar err error
:or by defining different names for the error:
With
ErrorOr
we don't need tricks:Inconsistent use of
(T, error)
There are APIs that are exception to 4 states of
(T, error)
rule, such as, theio.Reader.Read
. But they are the minority.The documentation for this interface method requires 4 paragraphs just to explain that this API can actually return a value and an error at the same time. If it were common practice in Go to return a value and an error at the same time, it would not be necessary 4 paragraphs of documentation to explain this notable exception.
Furthermore, the fact that it requires such as careful explanation is evidence in itself that this is a pitfall for API callers. And reading through the details it sounds very error prone and confusing.
A new type like
ErrorOr
could also be useful for a new API likeRead(...)ErrorOr[int]
because it would mean more consistency regarding error handling across APIs, we could also remove those 4 paragraphs of documentation, less pitfalls for developers, and also better type safety. API callers would need to either handle the value or the error, and there would be no ambiguity in that.Design ideas
One possible definition of
ErrorOr
is the following:The
New
andError
constructors only allow for either a value or error, they don't allow for both an error and a proper value.Related work
The
ErrorOr
has different names in other languages but in essence it's the same idea:Known issues
new(ErrorOr[T])
can create a state that is neither error not a proper value. Perhaps we would need special tooling or compiler support to prevent or produce a warning if a developer wrote this code, since it would be desirable to always use eithererroror.New
orerroror.Error
.Future ideas
This is out of scope for this proposal but I think it's an interesting, possible future extension of the
ErrorOr
idea, that can help with Go exceptions without actually requiring exceptions.Let's say we have a new operator (I will use
<-
but other syntactic choices are possible), then we could do the following:In this example,
OtherFunc
andOtherFunc2
are equivalent but inOtherFunc
we use this new operator to avoid having to explicitly write the error handling code. The new operator<-
does the automatic error handling for us by checking if theErrorOr
contains an error and if so, return that error to the caller. This may be an alternative to introducing exceptions without actually requiring full support for exceptions.Alternatives considered
An alternative to
ErrorOr
would be to treat(T, error)
like a tuple type so that a type likefunc()(T, error)
could be used to instantiate generic functions likefunc()T
. But this would be a much bigger change thanErrorOr
with far reaching implications for the language. TheErrorOr
is just a new type.More resources
Initial discussion was https://github.com/golang/go/discussions/48287#discussioncomment-2417886