Lifetimes

[MRECoelho]

Lifetimes

Somewhat important note

It might be relevant to point out that I'm a beginner myself and that the following content might need some reviewing. However, I think that by now I did develop the somewhat right mental model. All the knowledge I gained comes from the sources listed at References and, most of if not all, my examples are based on examples from these sources. That is my way of saying I'm not smart or creative enough to come up with my own examples. Before diving into lifetimes, I should mention that a highly requested feature is on its way to getting implemented. This is Non-Lexical Lifetimes (NLL). At the moment of writing this feature is available in the nightly build but is still under development. NLL loosens the lifetime constraints and might play an important role when working with lifetimes. I will, however, for the sake of simplicity completely ignore the possibilities of NLL for the following reasons. First of all, I think we need to establish the correct mental model of lifetimes before jumping into non-lexical lifetimes. We can always update our mental model when we get there. Secondly, my understanding of NLL at this moment is still insufficient and finally, the fact that it is still under development and future changes are possible, it might be confusing when trying to work on actual examples. Having this said, I do think we should discuss NLL at some point. NLL were requested because lifetimes can be too constrictive. I believe that if our mental model of lifetimes are correct we should be able to figure out its problematic constraints and conclude why or when we would need NLL. Anyway, for now let's forget about NLL and start with (lexical) lifetimes.

General concept of lifetimes

The rust compiler is provided with the (in)famous borrow checker. The borrow checker, as the name suggest, tries to determine whether all borrows are valid. This function prevents dangling pointers which could result in a 'use after free'. The Rust's borrow checker aims to prevent that problem by using lifetimes.

Quoting the Rustonomicon

"Rust enforces the borrowing rules through lifetimes. Lifetimes are effectively just names for scopes somewhere in the program. Each reference, and anything that contains a reference, is tagged with a lifetime specifying the scope it's valid for."

In other words the compiler (or actually borrow checker) needs to know/understand the lifetimes of references (or anythings that contains a reference). Now this is party, in my experience, why lifetimes can be confusing. Lifetimes are not really meant for us, they are meant for the borrow checker. To quote Daniel P. Clark:

"[...] the important thing to know is that you aren’t changing the behavior of lifetimes when you use them — they’re simply declaring them. That is, lifetimes with annotations work the same way as when there aren’t written annotations. They are simply markers to help clarify for the compiler the contexts for which the lifetimes are involved."

What is important to note is that by explicitly stating the lifetimes were not controlling the actual lifetimes but merely explicitly pointing them out. We control the lifetimes in the way we write our code and not by displaying the lifetime annotation.

Until now in the Learning Rust streams we haven't really played with lifetimes, but the were there all along. The Rust compiler allows us the elide lifetimes in all the cases when the compiler can it out by itself. The problems occur when the situation becomes ambiguous and such case the compiler demands explicit lifetimes. The elision of lifetimes enables us the write less and clear code and we will discuss this property more later on.

Reviewing the problem during the stream

It might be worth mentioning that in the Rust terminology we use the terms input lifetimes and output lifetimes in the case of functions, which make it a more convenient to talk about it. They refer respectively to the lifetimes of the input parameters and the return value. Now, let's review one of the problems from the stream, starting with the base case.

fn main(){
    let v1 = "hi";
    let v2 = String::from("world");
    let result = longest(a,b);
    println!("{}", result);
}

fn longest<'a>(x: &'a str, y: &'a str) -> &'a str {
    if x.len() > y.len() {
        x
    } else {
        y
    }
}

Looking at the snippet above, you can probably see that this will work, but let us workout what is happening with the lifetimes. Focusing on the function fn longest() we see that both input lifetimes have lifetimes 'a and the function returns a &str with lifetime 'a. In this example we see that the input- and output lifetimes are associated through lifetime 'a. Since we assign the returned value to the variable result we are basically saying that result may not outlive either variable x/v1 or y/v2. We can simply reason that if result would outlive its by-lifetime-associated variables v1 and/or v2 we would have a dangling pointer. That is, if v1 or v2 would go out of scope result might point to memory location that isn't 'valid'. Luckily here it is not the case and we can run the program without a problem.

Later in the stream we tried something similar to this:

fn main(){
    let v1 = "hi";
    let result;
    {
        let v2 = String::from("world");
        result = longest(v1,v2);

    }
    println!("{}", result);
}

fn longest<'a,'b>(x: &'a str, y: &'b str) -> &'a str {
    if x.len() > y.len() {
        x
    } else {
        y
    }
}

Purely looking at the snippet above, we can argue that this will fail since result is in fact a reference to either v1 or v2, but by now v2 is out of scope. The Rust compiler simply does not allow this behavior. So what about the lifetimes? Trying to add lifetime 'b does not help. First off all, at this moment we specified that the returned value has lifetime 'a and yet looking at the body of the function we might return a reference with lifetime 'b. This is simply not possible, but this is also not what we really want. What we actually want is that our returned value simple does not outlive out its references (since we now know this is illegal in Rust). So we are back to the previous example using only lifetime 'a. Now changing the lifetime to 'a will not help our case, but the problem does not lie with the specification of the lifetimes but simply with the way we structured our code. So either we change our code so variable v2 is still in scope when trying to access result or we we simply copy the entire structure and do not return a reference associated with variable v1 or v2 but a new String for example.

The text above might be quite verbose, but this was something I struggled with while it is actually not that complicated. My problem was that I though I had some influence when working with the lifetime parameters. I actually quit Rust a year ago because I couldn't figure some things out one of which was lifetimes.

During that same stream it was mentioned that multiple lifetimes are possible. Jim Blandy & Orendorff's 'Programming Rust' provides a fun example using lifetimes and structs, here is one from page 111:

struct S<'a>{
    x: &'a i32,
    y: &'a i32,
}

fn main(){
    let x = 10;
    let r;
    {
        let y =20;
        {
            let s = S{x: &x, y: &y};
            r =s.x;
        }
    }

}

The snippet aboves look quite alright, this will not compile. Eventhough r(a reference to x) does not outlive x nor does s outlive y we get the error that y does not live long enough. This may seem odd, but this can be explained by the following:

• Both fields of S are references with the same lifetime 'a, so Rust must find a single lifetime that works for both s.x and s.y.
• We assign r = s.x, requiring 'a to enclose r's lifetime. That is, s.x may not outlive r
• We initialized s.y to be a a reference to &y requiring 'a to be no longer than y's lifetime. That is, s.y may not outlive y

This is impossible to satisfy as no lifetime is shorter that the scope of y but longer than r.

The problem originates from us demanding both references in S to have the same lifetime 'a. An easy solution to this problem is shown below.

struct S<'a, 'b>{
    x: &'a i32,
    y: &'b i32,
}

fn main(){
    let x = 10;
    let r;
    {
        let y =20;
        {
            let s = S{x: &x, y: &y};
            r =s.x;
        }
    }

}

So again, we are not actually controlling the lifetimes but merely pointing them out. The actual lifetimes are defined by the structure of our program, we just help the borrow checker out and state explicitly what we mean. I hope this clears everything up, or at least give some insight into lifetimes. Let me know if more examples are needed. In the mean time I would advice you to read some of the references below. Each do a much better job at explaining this topic than I ever could. I just tried to give my perspective and stress out the points I was struggling with.

Lifetime elision

As mentioned before Rust allows lifetimes to be elided when encountering a non-ambiguous case, which saves us typing them and make the code less cluttered. Some examples straight from the Rust Book [link] are provided below.

fn print(s: &str); // elided
fn print<'a>(s: &'a str); // expanded
//even functions like print use it but luckily we don't need to provide the lifetime

fn debug(lvl: u32, s: &str); // elided
fn debug<'a>(lvl: u32, s: &'a str); // expanded

// In the preceding example, `lvl` doesn’t need a lifetime because it’s not a
// reference (`&`). Only things relating to references (such as a `struct`
// which contains a reference) need lifetimes.

fn substr(s: &str, until: u32) -> &str; // elided
fn substr<'a>(s: &'a str, until: u32) -> &'a str; // expanded

fn get_str() -> &str; // __ILLEGAL__, no inputs
// -> a reference to what? 

fn frob(s: &str, t: &str) -> &str; // __ILLEGAL__, two inputs
fn frob<'a, 'b>(s: &'a str, t: &'b str) -> &str; // Expanded: Output lifetime is ambiguous
// a valid case would be:
fn frob<'a>(s: &'a str, t: &'a str) -> &'a str;

Further more there is one rule with lifetimes we haven't seen so far and that on is:

• If there are many input lifetimes, but one of them is a reference to self, the lifetime of self is assigned to all elided output lifetimes.

So, self takes precedence over the other lifetimes if not explicitly stated otherwise. Other than that there is no real magic here. Just know that lifetimes are 'everywhere', but most of the time the compiler can figure out what we're trying to do.

Non-lexical lifetimes

(Not sure if NLL should be explained here or get it's own entry. Behind the scenes some pretty complex stuff is being done, see Niko Matsakis' post on this. However, for programmers it might just be a small update to our mental model. I'd suggest trying to get a more experienced programmer to comment on NLL)

References (in random order)

Understanding Lifetimes in Rust by Daniel P. Clark Understanding Rust: ownership, borrowing, lifetimes by Sergey Bugaev Understanding Lifetime in Rust – Part I by 'bibhas2' Understanding Lifetime in Rust – Part II by 'bibhas2' Programming Rust by Jim Blandy & Jason Orendorff

[jhagans3]

I do not know if this is what you were looking for during the stream (ch 19 Lifetime Subtyping):

fn main(){
    let v1 = "hi";
    let result;
    {
        let v2 = "world";
        result = longest(v1,v2);

    }
    println!("{}", result);
}

fn longest<'a,'b: 'a>(x: &'a str, y: &'b str) -> &'a str {
    if x.len() > y.len() {
        x
    } else {
        y
    }
}

https://play.rust-lang.org/?gist=52b34e11b26468bb2e9fc1510e19acbe&version=undefined&mode=undefined

[BrooksPatton]

Is this setting the type of the second lifetime to be the first? At least it looks likes its the same syntax as setting the type.

[MRECoelho]

@jhagans3 thank you for pointing that out. I was not aware of lifetime subtyping. I read the 'Advanced Lifetimes' chapter, but I thought they used a somewhat confusing example. I get the idea, but I'm a bit confused about the internals.

Anyway, I played around with @jhagans3 example and after using {:p} in the println!() function to see the memory location of the references and I stumbled somewhat on a 'aha'-moment. Displayed below is his code with my comments.

fn main(){
    let v1 = "hi"; // points to 0x...001
    let result;
    {
        let v2 = "world"; // points to 0x...002
        result = longest(v1,v2); // points to 0x...002
        // two variables point to the same memory location, but this is
        // not a problem since you can have multiple (non mutable) references
    }  // v2 goes out of scope
    println!("{}", result); // information at 0x...002 still exists as result points to it
}

fn longest<'a,'b: 'a>(x: &'a str, y: &'b str) -> &'a str {
    if x.len() > y.len() {
        x
    } else {
        y
    }
}

Looking at it this way, it makes sense. However, it also made me wonder if this is only possible in the case &str's that are known at compile time. Let's say you'd like to try this on some user input or maybe some random words from a file, would that still work? For example, I couldn't get the code below to work (rust version 1.26).

fn main(){
    let v1 = String::from("hi"); // neither could I get it to work with std::io::stdin().read_line()
    let result;
    {
        let v2 = String::from("world");
        result = longest(&v1,&v2);
    }
    println!("{}", result);
}

fn longest<'a,'b: 'a>(x: &'a str, y: &'b str) -> &'a str {
    if x.len() > y.len() {
        x
    } else {
        y
    }
}

At the moment I would argue that the above would never be possible because you need a &mut, meaning only one reference is allowed at one time (and thus only one mutable reference at the time).

This may look a bit off topic, but I'd like to confirm the idea I have on lifetimes (or proven wrong) which is that: "we can only use the magic of lifetime subtyping in the case were we know certain outcomes ahead of time but the compiler is having trouble seeing that". So we're just helping out the compiler but not in actual control of the lifetimes as defined in my original post.

[jhagans3]

@MRECoelho Yeah using str is kinda the easy way out

... the "string" literal. It can be assigned directly to a static variable without modification because its type signature:&'static str has the required lifetime of 'static. (https://doc.rust-lang.org/rust-by-example/custom_types/constants.html)

This works for the String type

fn main(){
    let v1 = String::from("hi");
    {
        let v2 = String::from("world");
        let result = longest(&v1,&v2);
        println!("{}", result);
    }
}

fn longest<'a,'b: 'a>(x: &'a str, y:&'b str) -> &'a str {
    if x.len() > y.len() {
        x
    } else {
        y
    }
}

result depends on v1 and v2 so it needs to live at most as long as 'b which is a subset of 'a, I think that explanation is correct. https://play.rust-lang.org/?gist=5a27b167e3d718d30c407640787c9626&version=stable&mode=debug

I am still learning this stuff too, this also helped me "see" the Linear/Affine types https://rufflewind.com/2017-02-15/rust-move-copy-borrow

[MRECoelho]

@jhagans3 Thanks for the reassurence! The rufflewind link is superb! I saw it before and it helped me some much visualizing the borrow system. I completely forgot about it. @BrooksPatton make sure to check it out!

A little update on my inital post Throughtout this post I'm stating that you have no actual control over lifetimes, which in retrospect is too strict and not entirely accurate. Both my wording and my mental model were a bit off. Thanks to @jhagans3 I learned quite a bit more about this topic and helped me with the latter. As far as wording goes, I'm currently working on a blog post which will be essentially a rewritten version of this thread. I'll let you know when it's finished and where to find it. For now, I think I'll leave it as it is (say for historic accuracy :) ). In the meantime, try to see the lifetime annotation as a way to control some flow or current. The structure of your code defines the sources and in that way you have control of where the sources are. Lifetime annotation merely helps guiding a certain flow or current. So when I mean that you have no control with lifetimes, I mean that lifetimes do not define these sources.