search

rust-lang / rust

Empowering everyone to build reliable and efficient software.
https://www.rust-lang.org
Other
94.9k stars 12.23k forks source link

Tracking issue for `?` operator and `try` blocks (RFC 243, `question_mark` & `try_blocks` features) #31436

Open nikomatsakis opened 8 years ago

nikomatsakis commented 8 years ago

Tracking issue for rust-lang/rfcs#243 and rust-lang/rfcs#1859.

Implementation concerns:

reem commented 8 years ago

The accompanying RFC discusses a desugaring based on labeled return/break, are we getting that too or will there just be special treatment for ? and catch in the compiler?

EDIT: I think labeled return/break is an excellent idea separate from ? and catch, so if the answer is no I will probably open a separate RFC for it.

nikomatsakis commented 8 years ago

Labeled return/break is purely for explanatory purposes.

On Fri, Feb 5, 2016 at 3:56 PM, Jonathan Reem notifications@github.com wrote:

The accompanying RFC discusses a desugaring based on labeled return/break, are we getting that too or will there just be special treatment for ? and catch in the compiler?

— Reply to this email directly or view it on GitHub https://github.com/rust-lang/rust/issues/31436#issuecomment-180551605.

glaebhoerl commented 8 years ago

Another unresolved question we have to resolve before stabilizing is what the contract which impls of Into have to obey should be -- or whether Into is even the right trait to use for the error-upcasting here. (Perhaps this should be another checklist item?)

petrochenkov commented 8 years ago

@reem

I think labeled return/break is an excellent idea ... I will probably open a separate RFC for it.

Please do!

thepowersgang commented 8 years ago

On the subject of the Carrier trait, here is a gist example of such a trait I wrote back early in the RFC process. https://gist.github.com/thepowersgang/f0de63db1746114266d3

petrochenkov commented 8 years ago

How this is treated during parsing?

struct catch {
    a: u8
}

fn main() {

    let b = 10;
    catch { a: b } // struct literal or catch expression with type ascription inside?

}
eddyb commented 8 years ago

@petrochenkov Well, the definition couldn't affect parsing, but I think we still have a lookahead rule, based on the second token after {, : in this case, so it should still be parsed as a struct literal.

petrochenkov commented 8 years ago

Also

let c1 = catch { a: 10 };
let c2 = catch { ..c1 }; // <--

struct catch {}
let c3 = catch {}; // <--

+ https://github.com/rust-lang/rfcs/issues/306 if (when!) implemented. It seems like there are no conflicts besides struct literals.

Given the examples above I'm for the simplest solution (as usual) - always treat catch { in expression positions as start of a catch block. Nobody calls their structures catch anyway.

durka commented 8 years ago

It would be easier if a keyword was used instead of catch.

bluss commented 8 years ago

This is the keywords list: http://doc.rust-lang.org/nightly/grammar.html#keywords

durka commented 8 years ago

@bluss yeah, I admit none of them are great... override seems like the only one that is close. Or we could use do, heh. Or a combination, though I don't see any great ones immediately. do catch?

bluss commented 8 years ago

do is the only one that seems to be close IMO. A keyword soup with do as prefix is a bit irregular, not similar to any other part of the language? Is while let a keyword soup as well? That one feels ok now, when you are used to it.

est31 commented 8 years ago

port try! to use ?

Can't ? be ported to use try! instead? This would allow for the use case where you want to get a Result return path, e.g. when debugging. With try! this is fairly easy, you just override the macro at the beginning of the file (or in lib/main.rs):

macro_rules! try {
    ($expr:expr) => (match $expr {
        Result::Ok(val) => val,
        Result::Err(err) => {
            panic!("Error occured: {:?}", err)
        }
    })
}

You will get a panic stack trace starting from the first occurrence of try! in the Result return path. In fact, if you do try!(Err(sth)) if you discover an error instead of return Err(sth), you even get the full stack trace.

But when debugging foreign libraries written by people who haven't implemented that trick, one relies on try! usage somewhere higher in the chain. And now, if the library uses the ? operator with hardcoded behaviour, getting a stacktrace gets almost impossible.

It would be cool if overriding try! would affect the ? operator as well.

Later on when the macro system gets more features you can even panic! only for specific types.

If this proposal requires an RFC please let me know.

rpjohnst commented 8 years ago

Ideally ? could just be extended to provide stack trace support directly, rather than relying on the ability to override try!. Then it would work everywhere.

durka commented 8 years ago

Stack traces are just one example (though a very useful one, seems to me). If the Carrier trait is made to work, maybe that can cover such extensions?

On Sun, Feb 7, 2016 at 4:14 PM, Russell Johnston notifications@github.com wrote:

Ideally ? could just be extended to provide stack trace support directly, rather than relying on the ability to override try!. Then it would work everywhere.

— Reply to this email directly or view it on GitHub https://github.com/rust-lang/rust/issues/31436#issuecomment-181118499.

est31 commented 8 years ago

Without wanting to speculate, I think that it could work, albeit with some issues.

Consider the usual case where one has code returning some Result<V,E> value. Now we would need to allow for multiple implementations of the Carrier trait to coexist. In order to not run into E0119, one has to make all implementations out of scope (possibly through different traits which are per default not imported), and when using the ? operator, the user is required to import the wished implementation.

This would require everybody, even those who don't want to debug, to import their wished trait implementation when using ?, there would be no option for a predefined default.

Possibly E0117 can be an issue too if wanting to do custom Carrier implementations for Result<V,E>, where all types are outside bounds, so libstd should provide already provide a set of implementations of the Carrier trait with the most used use cases (trivial implementation, and panic! ing implementation, perhaps more).

Having the possibility to override via a macro would provide a greater flexibility without the additional burden on the original implementor (they don't have to import their wished implementation). But I also see that rust never had a macro based operator before, and that implementing ? via a macro isn't possible if catch { ... } is supposed to work, at least not without additional language items (return_to_catch, throw, labeled break with param as used in RFC 243).

I am okay with any setup which enables one to get Result stacktraces with an Err return path, while having only to modify a very small amount of code, prefferably at the top of the file. The solution should also work unrelated to how and where the Err type is implemented.

rphmeier commented 8 years ago

Just to chime in on bikeshedding: catch in { ... } flows pretty nicely.

durka commented 8 years ago

catch! { ... } is another backcompat choice.

durka commented 8 years ago

Also, not that I expect this to change anything, but a note that this is going to break multi-arm macros that were accepting $i:ident ?, in the same way that type ascription broke $i:ident : $t:ty.

dgrunwald commented 8 years ago

Don't overdo the backwards compatibility, just treat catch as a keyword when followed by { (possibly only in expression position, but I'm not sure if that changes much compatibility-wise).

I can also imagine some possible problems that don't involve struct literals (e.g. let catch = true; if catch {}); but I prefer a minor breaking change over a more ugly syntax.

Didn't we have a for adding new keywords, anyways? We could offer some kind of from __future__ opt-in for new syntax; or specify a rust language version number on the command-line / in Cargo.toml. I highly doubt that in the long term, we'll be able to work with only those keywords that are already reserved. We don't want our keywords to have three different meanings each, depending on context.

glaebhoerl commented 8 years ago

I agree. This isn't even the first RFC where this has come up (https://github.com/rust-lang/rfcs/pull/1444 is another example). I expect it won't be the last. (Also default from https://github.com/rust-lang/rfcs/pull/1210, although it's not an RFC I'm in favor of.) I think we need to find a way to add honest-to-god keywords instead of trying to figure out how to ad-hoc hack the grammar for every new case.

rkjnsn commented 8 years ago

Wasn't the whole argument for not reserving several keywords prior to 1.0 that we'd definitely be introducing a way to add new keywords to the language backward compatibly (possibly by explicitly opting in), so there was no point? Seems like now would be a good time.

aturon commented 8 years ago

@japaric Are you interested in reviving your old PR and taking this on?

japaric commented 8 years ago

@aturon My implementation simply desugared foo? in the same way as try!(foo). It also only worked on method and function calls, i.e. foo.bar()? and baz()? work but quux? and (quux)? don't. Would that be okay for an initial implementation?

eddyb commented 8 years ago

@japaric What was the reason for restricting it to methods and function calls? Wouldn't parsing it be easier as a general postfix operator?

japaric commented 8 years ago

What was the reason for restricting it to methods and function calls?

easiest way (for me) to test the ? expansion

Wouldn't parsing it be easier as a general postfix operator?

probably

aturon commented 8 years ago

@japaric It'd probably be good to generalize it to a full postfix operator (as @eddyb is suggesting), but it's fine to land ? with the simple desugaring and then add catch later.

japaric commented 8 years ago

@aturon Alright, I'll look into the postfix version by next week if no one beats me to it :-).

japaric commented 8 years ago

rebased/updated my PR in #31954 :-)

est31 commented 8 years ago

What about support for providing stack traces? Is that planned?

hexsel commented 8 years ago

I hate to be the +1 guy, but stack traces have saved good chunks of my time in the past. Maybe on debug builds, and when hitting the error path, the ? operator could append the file/line to a Vec in Result? Maybe the Vec could be debug-only too?

And that could be either exposed or turned into part of the error description...

mitsuhiko commented 8 years ago

I keep running into the situation where I want to use try! / ? inside iterators returning Option<Result<T, E>>. Unfortunately that currently does not really work. I wonder if the carrier trait could be overloaded to support this or would that go into a more generic From instead?

mitsuhiko commented 8 years ago

@hexsel I really wish the Result<> type would carry a vec of instruction pointers in debug and ? would append to it. That way DWARF info could be used to build a readable stacktrace.

eddyb commented 8 years ago

@mitsuhiko But how could you create and pattern-match Result? It's just an enum atm.

As for the Option wrapping, I believe you want Some(catch {...}).

est31 commented 8 years ago

Currently, my habit right now is to do try!(Err(bla)) instead of return Err(), so that I can override the try macro later on with one that panics, in order to get a backtrace. It works well for me, but the code I deal with is very low level, and mostly originates the errors. I still will have to avoid ? if I use external code that returns Result.

mitsuhiko commented 8 years ago

@eddyb it would need language support to carry hidden values in addition that you need to manipulate by other means. I was wondering if it can be done in other ways but I can't see how. The only other way would have been a standardized error box that can have additional data on it, but there is no requirement to have boxed errors and most people don't do it.

eddyb commented 8 years ago

@mitsuhiko I can think of a new (default) method on the Error trait and TLS. The latter is used by sanitizers sometimes.

mitsuhiko commented 8 years ago

@eddyb that only works though if the Result can be identified and that requires it to be boxed or it will move around in memory as it passes upwards the stack.

eddyb commented 8 years ago

@mitsuhiko The TLS? Not really, you just need to be able to compare the error by-value.

eddyb commented 8 years ago

Or even just by type (with linking From inputs and outputs), how many concurrent errors you want stacktraces from will ever have to be propagated simultaneously?

I am against adding Result-specific compiler hacks, personally, when simpler solutions work.

mitsuhiko commented 8 years ago

@eddyb the error passes upwards the stack. What you want is the EIP at every stack level, not just where it originates. Also errors are a) currently not comparable and b) just because they compare equal does not mean they are the same error.

how many concurrent errors you want stacktraces from will ever have to be propagated simultaneously

Any error caught down and rethrown as different error.

I am against adding Result-specific compiler hacks, personally, when simpler solutions work.

I don't see how a simpler solution works but maybe I'm missing something there.

eddyb commented 8 years ago

You can save the instruction pointer at every ? and correlate it with the error type. "Any error caught down and rethrown as different error." But how would you preserve that information if it was hidden in Result?

mitsuhiko commented 8 years ago

But how would you preserve that information if it was hidden in Result?

You don't need to store that information in the result. What you do need to store though is a unique ID for the origin of the failure so you can correlate it. And because the error trait is just a trait and does not have storage, it could be stored in the result instead. The instruction pointer vec itself would by no means have to be stored in result, that could go to TLS.

One way would be that you invoke a method failure_id(&self) on the result and it returns an i64/uuid or something that identifies the origin of the failure.

This would need language support no matter what because what you need is that as the result passes upwards through the stack, the compiler injects an instruction to record the stack frame it falls through. So the return of a result would look different in debug builds.

eddyb commented 8 years ago

"the compiler injects an instruction to record the stack frame it falls through" - but ? is explicit, this is nothing like exceptions, or do you not like recording only the ? it passed through?

Anyway, if you manually unpack the error and then put it back in an Err, how would that ID even be kept?

eddyb commented 8 years ago

"And because the error trait is just a trait and does not have storage, it could be stored in the result instead" There is a variant on this: implementing the Error trait could be special-cased in the compiler to add an extra integer field to the type, creating the type would trigger an ID to be generated, and copy/drop would effectively increment/decrement the refcount (and eventually clear it from the TLS "in-flight error set" if Result::unwrap is not used).

But that would conflict with the Copy trait. I mean, so would your plan, adding any special behavior to Result that is not triggered by ? or other specific user actions can invalidate the Copy invariants.

EDIT: At this point you might as well embed an Rc<ErrorTrace> in there. EDIT2: What am I even saying, you can clear the associated error trace on catch. EDIT3: Actually, on drop, see below a better explanation.

mitsuhiko commented 8 years ago

"the compiler injects an instruction to record the stack frame it falls through" - but ? is explicit, this is nothing like exceptions, or do you not like recording only the ? it passed through?

That does not work because there are too many frames you can fall through which do not use ?. Let alone that not everybody is going to handle errors with just ?.

Anyway, if you manually unpack the error and then put it back in an Err, how would that ID be even kept?

That's why it would have to be compiler support. The compiler would have to track local variables that are results and do it's best to propagate the result id onwards for re-wraps. If this is too magical then it could be restricted to a subset of operations.

eddyb commented 8 years ago

That does not work because there are too many frames you can fall through which do not use ?. Let alone that not everybody is going to handle errors with just ?.

Okay, I could see returning Result directly be special-cased in complex functions with multiple return paths (some of which would be early returns from ?).

If this is too magical then it could be restricted to a subset of operations.

Or made entirely explicit. Do you have examples of non-? re-wrapping that would have to be magically tracked by the compiler?

mitsuhiko commented 8 years ago

@eddyb The common case of manual handling of errors is an IoError where you want to handle a subset:

loop {
  match establish_connection() {
    Ok(conn) => { ... },
    Err(err) => {
      if err.kind() == io::ErrorKind::ConnectionRefused {
        continue;
      } else {
        return Err(err);
      }
    }
  }
}
eddyb commented 8 years ago

@mitsuhiko Then keeping the ID inside io::Error would definitely work.

eddyb commented 8 years ago

So to recapitulate, a Vec<Option<Trace>> "sparse integer map" in TLS, keyed by struct ErrorId(usize) and accessed by 3 lang items:

If the transformation is done on MIR, Location can be computed from the Span of the instruction triggering either construction of an error value or writing to Lvalue::Return, which is much more reliable than an instruction pointer IMO (no easy way to get that in LLVM anyway, you'd have to emit inline asm for each specific platform).