Tracking issue for specialization (RFC 1210)

nikomatsakis commented 8 years ago

This is a tracking issue for specialization (rust-lang/rfcs#1210).

Major implementation steps:

[x] Land https://github.com/rust-lang/rust/pull/30652 =)
[ ] Restrictions around lifetime dispatch (currently a soundness hole)
[ ] default impl (https://github.com/rust-lang/rust/issues/37653)
[ ] Integration with associated consts
[ ] Bounds not always properly enforced (https://github.com/rust-lang/rust/issues/33017)
[ ] Should we permit empty impls if parent has no default members? https://github.com/rust-lang/rust/issues/48444
[ ] implement "always applicable" impls https://github.com/rust-lang/rust/issues/48538
[ ] describe and test the precise cycle conditions around creating the specialization graph (see e.g. this comment, which noted that we have some very careful logic here today)

Unresolved questions from the RFC:

Should associated type be specializable at all?
When should projection reveal a default type? Never during typeck? Or when monomorphic?
Should default trait items be considered default (i.e. specializable)?
Should we have default impl (where all items are default) or partial impl (where default is opt-in); see https://github.com/rust-lang/rust/issues/37653#issuecomment-616116577 for some relevant examples of where default impl is limiting.
How should we deal with lifetime dispatchability?

Note that the specialization feature as implemented currently is unsound, which means that it can cause Undefined Behavior without unsafe code. min_specialization avoids most of the pitfalls.

aturon commented 7 years ago

@afonso360 No, a separate issue is fine.

As a general point: at this point further work on specialization is blocked on the work on Chalk, which should allow us to tackle soundness issues and is also likely to clear up the ICEs being hit today.

sgrif commented 7 years ago

Can someone clarify if this is a bug, or something that is purposely forbidden? https://is.gd/pBvefi

rphmeier commented 7 years ago

@sgrif I believe the issue here is just that projection of default associated types is disallowed. Diagnostics could be better though: https://github.com/rust-lang/rust/issues/33481

sgrif commented 7 years ago

Could you elaborate on why it is expected to be disallowed? We know that no more specific impl could be added, since it would violate the orphan rules.

rphmeier commented 7 years ago

This comment indicates it's to necessary for some cases in order to require soundness (although I don't know why) and in others to force consumers of the interface to treat it as an abstract type: https://github.com/rust-lang/rust/blob/e5e664f/src/librustc/traits/project.rs#L41

sgrif commented 7 years ago

Was anyone ever able to look at https://github.com/rust-lang/rust/issues/31844#issuecomment-266221638 ? Those impls should be valid with specialization as far as I can tell. I believe there is a bug that is preventing them.

SergioBenitez commented 7 years ago

@sgrif I believe the issue with your code there may be similar to the issue in https://github.com/rust-lang/rust/issues/31844#issuecomment-284235369 which @withoutboats explained in https://github.com/rust-lang/rust/issues/31844#issuecomment-284268302. That being said, based on @withoutboats's comment, it seems that the present local reasoning should allow your example to compile, but perhaps I'm mistaken as to what's expected to work.

As an aside, I tried to implement the following, unsuccessfully:

trait Optional<T> {
    fn into_option(self) -> Option<T>;
}

impl<R, T: Into<R>> Optional<R> for T {
    default fn into_option(self) -> Option<R> {
        Some(self.into())
    }
}

impl<R> Optional<R> for Option<R> {
    fn into_option(self) -> Option<R> {
        self
    }
}

I intuitively expected Option<R> to be more specific than <R, T: Into<R>> T, but of course, nothing prevents an impl<R> Into<R> for Option<R> in the future.

I'm not sure why this is disallowed, however. Even if an impl<R> Into<R> for Option<R> was added in the future, I would still expect Rust to choose the non-default implementation, so as far as I can see, allowing this code has no implication on forward-compatibility.

In all, I find specialization very frustrating to work with. Just about everything I expect to work doesn't. The only cases where I've had success with specialization are those that are very simple, such as having an two impls that include T where T: A and T where T: A + B. I have a hard time getting other things to work, and the error messages don't indicate why attempts to specialize don't work. Of course, there's still a road ahead, so I don't expect very helpful error messages. But there seem to be quite a few cases where I really expect something to work (like above) but it just doesn't, and it's currently quite difficult for me to ascertain if that's because I've misunderstood what's allowed (and more importantly, why), if something is wrong, or if something just hasn't been implemented yet. A nice overview of what's going on with this feature as it stands would be very helpful.

burns47 commented 7 years ago

I'm not positive this is in the right place, but we ran into a problem on the users forum that I'd like to mention here.

The following code (which is adapted from the RFC here) does not compile on nightly:

#![feature(specialization)]

trait Example {
    type Output;
    fn generate(self) -> Self::Output;
}

default impl<T> Example for T {
    type Output = Box<T>;
    fn generate(self) -> Self::Output { Box::new(self) }
}

impl Example for bool {
    type Output = bool;
    fn generate(self) -> Self::Output { self }
}

This doesn't really seem like a glitch but more like a usability problem - if a hypothetical impl specialized only the associated type in the example above, the defaulti impl of generate wouldn't typecheck.

Link to the thread here

dtolnay commented 7 years ago

@burns47 there is a confusing but useful workaround here: https://github.com/rust-lang/rust/issues/31844#issuecomment-263175793.

burns47 commented 7 years ago

@dtolnay Not quite satisfactory - what if we're specializing on traits we don't own (and can't modify)? We shouldn't need to rewrite/refactor trait definitions to do this IMO.

bstrie commented 7 years ago

Can anyone comment as to whether the code in the following issue is intentionally rejected? https://github.com/rust-lang/rust/issues/45542

ghost commented 6 years ago

Would specialization allow adding something like the following to libcore?

impl<T: Ord> Eq for T {}

impl<T: Ord> PartialEq for T {
    default fn eq(&self, other: &Self) -> bool {
        self.cmp(other) == Ordering::Equal
    }
}

impl<T: Ord> PartialOrd for T {
    default fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
        Some(self.cmp(other))
    }
}

This way you could implement Ord for your custom type and have Eq, PartialEq, and PartialOrd be automatically implemented.

Note that implementing Ord and simultaneously deriving PartialEq or PartialOrd is dangerous and can lead to very subtle bugs! With these default impls you would be less tempted to derive those traits, so the problem would be somewhat mitigated.

Alternatively, we modify derivation to take advantage of specialization. For example, writing #[derive(PartialOrd)] above struct Foo(String) could generate the following code:

impl PartialOrd for Foo {
    default fn partial_cmp(&self, other: &Foo) -> Option<Ordering> {
        self.0.partial_cmp(&other.0)
    }
}

impl PartialOrd for Foo where Foo: Ord {
    fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
        Some(self.cmp(other))
    }
}

This way the default impl gets used if Ord is not implemented. But if it is, then PartialOrd relies on Ord. Unfortunately, this doesn't compile: error[E0119]: conflicting implementations of trait `std::cmp::PartialOrd` for type `Foo`

scottmcm commented 6 years ago

@stjepang I certainly hope the blankets like that can be added -- impl<T:Copy> Clone for T too.

jan-hudec commented 6 years ago

I think

impl<T: Ord> PartialEq for T

should be

impl<T, U> PartialEq<U> for T where T : PartialOrd<U>

because PartialOrd requires PartialEq and can provide it too.

burdges commented 6 years ago

Right now, one cannot really use associated types to constrain a specialization, both because they cannot be left unspecified and because they trigger uneeded recursion. See https://github.com/dhardy/rand/issues/18#issuecomment-358147645

Centril commented 6 years ago

Eventually, I'ld love to see what I'm calling specialization groups with the syntax proposed by @nikomatsakis here https://github.com/rust-lang/rust/issues/31844#issuecomment-249355377 and independently by me. I'ld like to write an RFC on that proposal later when we're closer to stabilizing specialization.

nikomatsakis commented 6 years ago

Just in case nobody saw it, this blog post covers a proposal to make specialization sound in the face of lifetime-based dispatch.

earthengine commented 6 years ago

As copy closures were already stablized in Beta, developers have more motivation to stabilizing on specialization now. The reason is that Fn and FnOnce + Clone represent two overlapping set of closures, and in many case we need to implement traits for both of them.

Just figure out that the wording of rfc 2132 seems to imply that there are only 5 types of closures:

FnOnce (a move closure with all captured variables being neither Copy nor Clone)
FnOnce + Clone (a move closure with all captured variables being Clone)
FnOnce + Copy + Clone (a move closure with all captured variables being Copy and so Clone)
FnMut + FnOnce (a non-move closure with mutated captured variables)
Fn + FnMut + FnOnce + Copy + Clone (a non-move closure without mutated captured variables)

So if specification is not available in the near future, maybe we should update our definition of Fn traits so Fn does not overlapping with FnOnce + Clone?

I understand that someone may already implemented specific types that is Fn without Copy/Clone, but should this be deprecated? I think there is always better way to do the same thing.

glandium commented 6 years ago

Is the following supposed to be allowed by specialization (note the absence of default) or is it a bug?

#![feature(specialization)]
mod ab {
    pub trait A {
        fn foo_a(&self) { println!("a"); }
    }

    pub trait B {
        fn foo_b(&self) { println!("b"); }
    }

    impl<T: A> B for T {
        fn foo_b(&self) { println!("ab"); }
    }

    impl<T: B> A for T {
        fn foo_a(&self) { println!("ba"); }
    }
}

use ab::B;

struct Foo;

impl B for Foo {}

fn main() {
    Foo.foo_b();
}

without specialization, this fails to build with:

error[E0119]: conflicting implementations of trait `ab::B` for type `Foo`:
  --> src/main.rs:24:1
   |
11 |     impl<T: A> B for T {
   |     ------------------ first implementation here
...
24 | impl B for Foo {}
   | ^^^^^^^^^^^^^^ conflicting implementation for `Foo`

gnzlbg commented 6 years ago

@glandium what on earth is going on there? Nice example, here the playground link: https://play.rust-lang.org/?gist=fc7cf5145222c432e2bd8de1b0a425cd&version=nightly&mode=debug

nikomatsakis commented 6 years ago

@glandium that is https://github.com/rust-lang/rust/issues/48444

glandium commented 6 years ago

is it? there is no empty impl in my example.

MoSal commented 6 years ago

@glandium

 impl B for Foo {}

gnzlbg commented 6 years ago

@MoSal but that impl "isn't empty" since B adds a method with a default implementation.

alexreg commented 6 years ago

@gnzlbg It is empty by definition. Nothing between the braces.

MoSal commented 6 years ago


#![feature(specialization)]

use std::borrow::Borrow;

#[derive(Debug)]
struct Bla {
    bla: Vec<Option<i32>>
}

// Why is this a conflict ?
impl From<i32> for Bla {
    fn from(i: i32) -> Self {
        Bla { bla: vec![Some(i)] }
    }
}

impl<B: Borrow<[i32]>> From<B> for Bla {
    default fn from(b: B) -> Self {
        Bla { bla: b.borrow().iter().map(|&i| Some(i)).collect() }
    }
}

fn main() {
    let b : Bla = [1, 2, 3].into();
    println!("{:?}", b);
}

error[E0119]: conflicting implementations of trait `std::convert::From<i32>` for type `Bla`:
  --> src/main.rs:17:1
   |
11 | impl From<i32> for Bla {
   | ---------------------- first implementation here
...
17 | impl<B: Borrow<[i32]>> From<B> for Bla {
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ conflicting implementation for `Bla`
   |
   = note: upstream crates may add new impl of trait `std::borrow::Borrow<[i32]>` for type `i32` in future versions

Wouldn't specialization prevent possible future conflicts?

alexreg commented 6 years ago

Goodness me, this is a slow-moving feature! No progress in over two years, it seems (certainly according to the original post). Has the lang team abandoned this?

Centril commented 6 years ago

@alexreg see http://aturon.github.io/2018/04/05/sound-specialization/ for the latest development.

mark-i-m commented 6 years ago

@alexreg It turns out soundness is hard. I believe there is some work on the "always applicable impls" idea currently happening, so there is progress. See https://github.com/rust-lang/rust/pull/49624. Also, I believe that the chalk working group is working on implementing the "always applicable impls" idea too, but I don't know how far that has gotten.

Lymia commented 6 years ago

After a bit of wrangling, it seems it is possible to effectively implement intersection impls already via a hack using specialization and overlapping_marker_traits.

https://play.rust-lang.org/?gist=cb7244f41c040db41fc447d491031263&version=nightly&mode=debug

Centril commented 6 years ago

cc https://github.com/rust-lang/rfcs/pull/2532

real-felix commented 6 years ago

I tried to write a recursive specialized function to implement an equivalent to this C++ code:

C++ code

``` #include #include template size_t count(T elem) { return 1; } template size_t count(std::vector vec) { size_t n = 0; for (auto elem : vec) { n += count(elem); } return n; } int main() { auto v1 = std::vector{1, 2, 3}; assert(count(v1) == 3); auto v2 = std::vector{ std::vector{1, 2, 3}, std::vector{4, 5, 6} }; assert(count(v2) == 6); return 0; } ```

I tried this:

Rust code

``` #![feature(specialization)] trait Count { fn count(self) -> usize; } default impl Count for T { fn count(self) -> usize { 1 } } impl Count for T where T: IntoIterator, T::Item: Count, { fn count(self) -> usize { let i = self.into_iter(); i.map(|x| x.count()).sum() } } fn main() { let v = vec![1, 2, 3]; assert_eq!(v.count(), 3); let v = vec![ vec![1, 2, 3], vec![4, 5, 6], ]; assert_eq!(v.count(), 6); } ```

But I am getting an: ``` overflow evaluating the requirement `{integer}: Count` ``` I do not think that this should happen because `impl Count for T where T::Item: Count` should not overflow. EDIT: sorry, I just saw that [this was already mentioned](https://github.com/rust-lang/rust/issues/31844#issuecomment-358148461)

TeXitoi commented 6 years ago

@Boiethios Your usecase is working if you default on the fn and not on the impl:

#![feature(specialization)]

trait Count {
    fn count(self) -> usize;
}

impl<T> Count for T {
    default fn count(self) -> usize {
        1
    }
}

impl<T> Count for T
where
    T: IntoIterator,
    T::Item: Count,
{
    fn count(self) -> usize {
        let i = self.into_iter();

        i.map(|x| x.count()).sum()
    }
}

fn main() {
    let v = vec![1, 2, 3];
    assert_eq!(v.count(), 3);

    let v = vec![vec![1, 2, 3], vec![4, 5, 6]];
    assert_eq!(v.count(), 6);
}

alexreg commented 6 years ago

Has the soundness hole still not been fixed yet?

mark-i-m commented 6 years ago

@alexreg I don't think so. See http://smallcultfollowing.com/babysteps/blog/2018/02/09/maximally-minimal-specialization-always-applicable-impls/

My guess is that everyone's focused on the edition right now...

alexreg commented 6 years ago

Okay thanks... seems like this issue is dragging on forever, but fair enough. It's tough, I know. And attention is directed elsewhere right now unfortunately.

pythonesque commented 6 years ago

Can someone more concretely explain the rationale behind not allowing projections for default associated types in fully-monomorphic cases? I have a use case where I would like that functionality (in particular, it would be semantically incorrect for the trait to ever be invoked with types that weren't fully monomorphic), and if there's no soundness issue I don't completely understand why it's disallowed.

sgrif commented 6 years ago

@pythonesque There's some discussion at https://github.com/rust-lang/rust/pull/42411

pythonesque commented 6 years ago

Ah, I understand if it turns out that projection interacts badly with specialization in general. . And it is indeed true that what I want is of a "negative reasoning" flavor (though closed traits would not really be sufficient).

Unfortunately, I'm not sure if there's really any way to do what I want without such a feature: I'd like to have an associated type that outputs "True" when two passed-in types implementing a particular trait are syntactically equal, and "False" when they aren't (with the "False" case triggering a more expensive trait search which can decide whether they are "semantically" equal). The only real alternative seems (to me) to be to just always do the expensive search; which is fine in theory, but it can be a lot more expensive.

(I could work around this if the trait were intended to be closed, by just enumerating every possible pair of constructors in the head position and having them output True or False; but it's intended to be open to extension outside the repository, so that can't possibly work, especially since implementations in two different user repositories wouldn't necessarily know about each other).

Anyway, maybe this is just an indication that what I want to do is a bad fit for the trait system and I should switch to some other mechanism, like macros :P

gnzlbg commented 6 years ago

And it is indeed true that what I want is of a "negative reasoning" flavor (though closed traits would not really be sufficient).

An alternative to negative reasoning is requiring that a type implements only one trait of a closed set of traits, such that implementations with other other traits in the set cannot overlap (e.g. T implements one of { Float | Int | Bool | Ptr }).

pythonesque commented 6 years ago

Even if there were a way to enforce that in Rust (which there isn't, AFAIK?), I do not think that would solve my problem. I would like users in different crates to be able to implement an arbitrary number of new constants, which should compare equal only to themselves and unequal to every other defined constant, including ones unknown at crate definition time. I don't see how any closed set of traits (or even set of families of traits) can accomplish that goal by itself: this is a problem that fundamentally can't be solved without looking directly at the types. The reason it would be workable with default projections is that you could default everything to "don't compare equal" and then implement equality of your new constant to itself in whatever crate you defined the constant in, which wouldn't run afoul of orphan rules because all the types in the trait implementation were in the same crate. If I wanted almost any such rule but equality, even this wouldn't work, but equality is good enough for me :)

rmanoka commented 6 years ago

On present nightly, this works:

trait Foo {}
trait Bar {}

impl<T: Bar> Foo for T {}
impl Foo for () {}

but even with specialization, and using nightly, this does not:

#![feature(specialization)]

trait Foo<F> {}
trait Bar<F> {}

default impl<F, T: Bar<F>> Foo<F> for T {}
impl<F> Foo<F> for () {}

Does this have a rationale or is it a bug?

Boscop commented 6 years ago

@rmanoka Isn't this just the normal orphan rules? In the first case, no downstream crate could impl Bar for () so the compiler allows this, but in the second example, a downstream crate could impl Bar<CustomType> for () which would conflict with your default impl.

rmanoka commented 6 years ago

@Boscop In that scenario, the default impl should anyway be overridden by the non-default one below. For instance, if I had: impl Bar<bool> for () {} added before the other impls, then I would expect it to work (as per RFC / expectation). Isn't that correct?

Digging deeper along the lines of the counter-example you've mentioned, I realise (or believe) that the example satisfies the "always-applicable" test, and may be being worked on.

fogti commented 5 years ago

This issue probably depends on #45814.

Gladdy commented 5 years ago

Are there any plans to support trait bounds on the default that are not present in the specialisation?

As an example for which this would be very useful, such that you can easily compose handling of different types by creating a generic Struct with arbitrary Inner for the functionality that shouldn't be shared.

#![feature(specialization)]
trait Handler<M> {
    fn handle(&self, m:M);
}

struct Inner;
impl Handler<f64> for Inner {
    fn handle(&self, m : f64) {
        println!("inner got an f64={}", m);
    }
}

struct Struct<T>(T);
impl<T:Handler<M>, M:std::fmt::Debug> Handler<M> for Struct<T> {
    default fn handle(&self, m : M) {
        println!("got something else: {:?}", m);
        self.0.handle(m)
    }
}
impl<T> Handler<String> for Struct<T> {
    fn handle(&self, m : String) {
        println!("got a string={}", m);
    }
}
impl<T> Handler<u32> for Struct<T> {
    fn handle(&self, m : u32) {
        println!("got a u32={}", m);
    }
}

fn main() {
    let s = Struct(Inner);
    s.handle("hello".to_string());
    s.handle(5.0 as f64);
    s.handle(5 as u32);
}

Furthermore, in the example above, something odd that I've experienced - after removing the trait bound on the default Handler impl (and also self.0.handle(m)) the code compiles without issues. However, when you remove the implementation for u32, it seems to break the other trait deduction:

#![feature(specialization)]
trait Handler<M> {
    fn handle(&self, m:M);
}

struct Struct<T>(T);
impl<T, M:std::fmt::Debug> Handler<M> for Struct<T> {
    default fn handle(&self, m : M) {
        println!("got something else: {:?}", m);
    }
}
impl<T> Handler<String> for Struct<T> {
    fn handle(&self, m : String) {
        println!("got a string={}", m);
    }
}
// impl<T> Handler<u32> for Struct<T> {
//     fn handle(&self, m : u32) {
//         println!("got a u32={}", m);
//     }
// }
fn main() {
    let s = Struct(());
    s.handle("hello".to_string());
    s.handle(5.0 as f64);
}

Even though there is no code calling the handler for u32, the specialisation not being there causes the code to not compile.

PieterPenninckx commented 5 years ago

Edit: this seems to be the same as the second problem ("However, when you remove the implementation for u32, it seems to break the other trait deduction") that Gladdy mentioned a one post back.

With rustc 1.35.0-nightly (3de010678 2019-04-11), the following code gives an error:

#![feature(specialization)]
trait MyTrait<T> {
    fn print(&self, parameter: T);
}

struct Message;

impl<T> MyTrait<T> for Message {
    default fn print(&self, parameter: T) {}
}

impl MyTrait<u8> for Message {
    fn print(&self, parameter: u8) {}
}

fn main() {
    let message = Message;
    message.print(1_u16);
}

error:

error[E0308]: mismatched types
  --> src/main.rs:20:19
   |
18 |     message.print(1_u16);
   |                   ^^^^^ expected u8, found u16

However, the code compiles and works when I omit the impl MyTrait<u8> block:

#![feature(specialization)]
trait MyTrait<T> {
    fn print(&self, parameter: T);
}

struct Message;

impl<T> MyTrait<T> for Message {
    default fn print(&self, parameter: T) {}
}

/*
impl MyTrait<u8> for Message {
    fn print(&self, parameter: u8) {}
}
*/

fn main() {
    let message = Message;
    message.print(1_u16);
}

Is this by design, is this because the implementation is incomplete, or is this a bug?

Also, I would like to know if this use case for specialization (implementing traits with overlapping type parameters for a single concrete type as opposed to implementing the same trait for overlapping types) will be supported. Reading section "Defining the precedence rules" in RFC 1210, I think it would be supported, but the RFC does not give such examples and I don't know if we are still strictly following this RFC.

updogliu commented 5 years ago

Report a weirdness:

trait MyTrait {}
impl<E: std::error::Error> MyTrait for E {}

struct Foo {}
impl MyTrait for Foo {}  // OK

// But this one is conflicting with error message:
//
//   "... note: upstream crates may add new impl of trait `std::error::Error` for type
//    std::boxed::Box<(dyn std::error::Error + 'static)>` in future versions"
//
// impl MyTrait for Box<dyn std::error::Error> {}

Why is Box<dyn std::error::Error> peculiar (avoid using word "special") in this case? Even if it impls std::error::Error in the future, the impl MyTrait for Box<dyn std::error::Error> is still a valid specialization of impl<E: std::error::Error> MyTrait for E, no?

bjorn3 commented 5 years ago

is still a valid specialization

In your case the impl<E: std::error::Error> MyTrait for E can not be specialized, as it doesnt have any default methods.

RustyYato commented 5 years ago

@bjorn3 This looks like it should work, but it doesn't even if you add in dummy methods

in crate bar

pub trait Bar {}
impl<B: Bar> Bar for Box<B> {}

In crate foo

#![feature(specialization)]

use bar::*;

trait Trait {
    fn func(&self) {}
}

impl<E: Bar> Trait for E {
    default fn func(&self) {}
}

struct Foo;
impl Trait for Foo {}  // OK

impl Trait for Box<dyn Bar> {} // Error error[E0119]: conflicting implementations of trait

Note that if you change crate bar to

pub trait Bar {}
impl<B: ?Sized + Bar> Bar for Box<B> {}

Then crate foo compiles.

rust-lang / rust

Tracking issue for specialization (RFC 1210) #31844