"Official Orphans"

[Ixrec]

Sometimes, there's an impl that probably should exist, and client code probably should not have to provide it via a newtype, but neither the type's crate or the trait's crate is a good home for it. But we still want there to be a single, quasi-"official" home for these impls. As far as I know, Rust has no good answer for this case today.

This issue is for gathering examples of such use cases, and possibly discussing what a targeted solution for it might look like.

The only example I'm currently aware of is a hypothetical diesel-chrono crate.

[Ixrec]

I just noticed that withoutboats thinks mutually exclusive traits are part of the solution (or at least he did back in 2016). That seems significant.

[gbutler69]

One possibility for "solving" this diesel-chrono type of issue is the notion of "friend" crates.

Overview

• A crate containing some type S may declare a friend crate for S (or the mod containing the definition of S) with permission/responsibility for implementing Trait T for S. This "declaration" may be in a top-level file for the crate, something like, lib.rs/lib-friends.rs, main.rs/main-friends.rs (more about this in the details)
• An additional visibility modifier will be created, pub(friend), that makes items of a module "publicized" (i.e. from the "friends" perspective the item is public, but, can not be "pub" exported or "pub(friend)" exported from the "friend" crate) to any "friends" for S declared in the top-level (*-friends.rs) file
• Another visibility modifer will be created, pub(nofriend), equivalent to private for non-friend crates, that can be used inside pub(friend) items to make components not visible to friends
• It is a compiler error to both impl Trait T for S inside the crate and declare that crate C is a friend implementer with respect to T for S
• It is a compiler error to declare more than 1 friend for a module or struct (or other type) with respect to the same Trait
• It would be a breaking semver change to remove a "friend" declaration. It would not be a breaking change to add a "friend" declaration to a crate
• It would be a breaking "friend-level" semver change to change an existing pub(friend) item in a way (especially removal or removal of the pub(friend) to "private" but not pub(friend) to pub) that would be considered a breaking public-level semver change for any pub item. Breaking public vs friend semver numbering would be extended as follows:
  • pub semver would remain as it is
  • new "friend" semver would be added that would be the same as the pub semver with an additional component:
    • pub semver is: Major.Minor.Bugfix
    • friend semver is: Major.FriendMajor.Minor.FriendMinor.FriendBugfix
      • the Major.Minor parts should match with pub semver AT ALL TIMES
      • the FriendMajor should be incremented for a breaking "friend-level" API change
      • the FriendMinor should be incremented for a non-breaking "friend-level" API change
      • if the Major is incremented, FriendMajor, Minor, and FriendBugFix should reset to zero
      • if the Minor is incremented or zeroed, FriendMinor and FriendBugFix should reset to zero
      • if FriendMajor is incremented or zeroed, FriendMinor and FriendBugFix should zero
      • if FriendMinor is incremented or zeroed, FriendBugFix should zero
    • just like cargo tooling can be/is already developed for alerting when changes to public API should cause a semver bump, tooling can similarly support the same for friend semver
    • a "friend" crate must declare dependency to the "friending" crate by way of the "friend" semver rather than the "pub" semver
  • it is not required that a crate that declares top-level "friends" pub(friend) any items (in effect, the "friend" publicized API surface is identical to the "public" publicized API surface, in that case, FriendMajor/FriendMinor/FriendBugfix would always be 0
  • from the perspective of the Trait T crate, the impls provided by the "friend" create of the type S crate are equivalent for purposes of orphan/overlap rules
  • from the perspective of the type S create the impls provided by the "friend" crate do not exist
  • from the perspective of a crate that wishes to use the impls of T provided by the the friend crate for the type S, the impls are equivalent to impls provided by the crate with type S aside from where they are being "used" from
    • a friend crate accesses the "friended" items in the "friending" crate using a top-level implicit "friend" module
    • so if mod foo::bar is designated as friended to a particular crate, that module is referenced from the friend crate as "::crate-name::friend::foo::bar" instead of "::crate-name::foo::bar" (this allows internal re-organization of internal module/item hierarchy without breaking the "friended" crate (more about this in the details)

What does it look like?

// Crate t containing Trait T

// (nothing new/special required)

// Crate u containing trait U

// (nothing new/special required)

// Crate s providing structs that need impls for u::U and t::T

// main.rs/lib.rs
// (nothing new/special required)

// Make the 'baz' module's private (no visibility spec) (not pub(self), pub(nofriend), pub(super), pub(in ...mod)) items publicized to friends
pub(friend) mod baz;

...

// foo.rs (module foo)

// if nothing is designated as pub(friend) then the "friend" visible items are the same as the "pub" visible items 
// no changes to this module in that case

// foo/bar.rs (module foo::bar)

// even though this module isn't pub(friend) items within it (including structs, enums, Trait impls, fns, Inheren Methods, and sub-modules) can be pub(friend)

pub(friend) struct FooBarYep {
  // struct private elements are made "friend" accessible too
}

pub(friend) impl for FooBarYep {
  ...
}

pub(friend) fn FooBarYepProcessor( p : FooBarYep ) {

}

// not accessible to friends
struct FooBarNope {

}

// ...etc...

// baz.rs (module baz)

// nothing special needed here as the entire 'baz' module has been publicized to friends, so, if anything needs special access by a friend, it has it

// this is accessible to friends
struct Bazup {
  // all private/pub members of this accessible to friends
}

// however, if something should be "true private" (even hidden from friends, it can be declared as "pub(self)" or "pub(nofriend)"

// not accessible to friends
pub(self) struct Bazzit {
}

pub(nofriend) struct Bazout {
}

// accessible to friends 
struct Bazme {
    x : i32; // accessible to friends (because the mod has been "friended")
    pub(self) y: i32; // not accessible to friends, only accessible within the Baz mod
    pub(nofriend) z: i32; // not accessible to friends, true private even though the module was friended
    pub t : i32; // also accessible to friends
    pub(super) u : i32; // not accessible to friends
    pub(friend) v : i32; // accessible to friends (lint warning for unneeded pub(friend)
    pub(self,friend) v : 32; // only accessible to self and friends
}

// accessible to friends, but, has a lint warning for unneeded pub(friend)
pub(friend) Bazyou {
  ...
}

// main-friends.rs/lib-friends.rs (declare friend crates)

friend friend-t-impl-s-Sy impl (::t::T) for ::crate::{Baz,Foo::Bar} // provide impls of Trait T of crate t by friend crate friend-t-impl-s-Sy
friend friend-t-impl-s-Sy impl (::t::T2) for ::crate::{Baz,Foo::Bar} // provide impls of Trait T2 of crate t by friend crate friend-t-impl-s-Sy

//// friend friend-t-impl-s-Sy impl (::t::T,::t::T2) for ::crate::{Baz,Foo::Bar}

friend friend-u-impl-s-Sy impl (::u::U) for ::crate::{Baz,Foo::Bar} // provide impls of Trait U of crate u by friend crate friend-u-impl-s-Sy

// Crate friend-t-impl-s-Sy containing impls for Trait T from t for types Sy subset(a) of Sx of crate s 

use ::t::T;
use ::u::U;

use ::s::friend::Baz;
use ::s::friend::Foo::Bar;

pub timpl {
  impl T for Baz::Bazup {
  ...
  }

  impl T for Bar::FooBarYep {
  ...
  }

  // Not permitted
  //impl T for Bar::FooBarNope {
  // ...
  //}
}

// Crate friend-u-impl-s-Sz containing impls for Trait U from u for types Sz subset(b) of Sx of crate s 

// similar for u::U as for t::T above

// Crate u that uses Sy from s and wants to use the impls of T for Sy from s provided by friend-t-impl-s-Sy and for Sz from s provided by frient-t-impl-s-Sz

use ::t::T;
use ::u::U;
use ::s::Bar::FooBarYep;
use ::friend-t-impl-s-Sy::timpl;

// have impl of T for FooBarYep provided by friend-t-imp-s-Sy crate here

How do we teach this?

• update TRPL
• add nomicon entry/chapter/section
• (more to come if actual RFC is made from this brainstorm proposal)

What does this give us?

• consistent overlap/orphan rules (coherence/1 and only 1 implementation of T for S)
• crate t of Trait T need have no knowledge of crate s of S or crate friend of s for S for t of T
• internals of crate s need not be aware of crate t For T or friend crates that provide t for T impls
• maintainer of crate s need not worry about details of how trait T of crate t is implemented
• allows maintainer of crate s to allow known "friends" to have more access than "pub" (optionally)
• easy PR (just add a line to the *-friends.rs) to add support of 3rd party impl of 2nd party traits, no further knowledge of anything beyond that required in crate s
• clear compile errors if crate s tries to both impl trait T of t and declare a friend that will provide such impl
• allows crate s to easily control which 3rd party crate will supply impls of a given trait to a given module
• a "friend" crate is not a dependency (will not need downloaded/compiled in order to compile crate s)
• a "trait" crate that crate s is not providing impls itself for is not a dependency of crate s
• a way for exposed "friend" and "public" interfaces of crate s to evolve semi independently

What DOESN'T this give us?

• crate s has no knowledge of the existence and need have no reference to 2nd party crate t that provide T or 3rd party friend crate the provides impls of T
• a way to add 3rd party impls for new traits to existing crate without any cooperation from the maintainers of crate s
• a way to have multiple independent 3rd party impls of T in t for S in s

What are the problems/uncertainties?

• Does "friend" visibility violate encapsulation? Can it lead to unexpected unsafety/UB? If so, can this proposal work without the notion of "friend" visibility usefully (can any proposal)?
• What if crate s doesn't want to add "friend" declarations and doesn't want to add impls for traits? Is that a problem?
• How much knowledge of specific trait impl's are required to make correct/useful use of "friend" visibility in crate s? Does it matter?

How would the compiler implement this?

• (more to follow if this develops beyond the "brainstorm" stage)

How would cargo implement this?

• (more to follow if this develops beyond the "brainstorm" stage)

Alternatives

• No "friend" visibility specifier
• alternative names (instead of "friend" perhaps "impl" or something similar)?

[crlf0710]

serde-json is another example. While it's the defacto standard crate for json types(and quite satisfactory), it is biased for serde usages, and not a neutral representation crate as it should be.

serde-xml is in similar situation (And might be even worse).

[jplatte]

I've got two features in the js_int crate, serde and rocket_04, that only serve to add trait implementations. I would argue that the latter would make a bit more sense in its own crate (the former less so because serde feels pretty 'fundamental').

[luxalpa]

I've got two use cases from my projects: speedy + glam and chrono / index_vec + bitcode. The latter one currently has some escape hatch (where bitcode just allows deferring to serde), but the former one is really annoying as glam is being used all over the project and new typing it is just a very big deal (currently using git subtree on glam instead).