Some problems have raised naturally about the decision to have the rules in the core language.
(See also a somewhat related issue #22.)
What
The initial problem is that, the so-called [in]coherence is never well-defined besides the language rules themselves, if the non-existence of coherence has ever been a problem.
This is a chicken-and-egg problem. It is doubtful whether a formal definition can be precisely established in this context. Probably some informal intents still make sense. However, I see few people respect the background problems sufficiently; many users insists on the necessity of the so-called coherence, just because it sounds goodTM plus totally lacking of such properties is obviously horrible, intuitively (so we better have it).
Informally, the so-called coherence is generally (not specifically to Rust) a group of properties about the identity of entities introduced in arbitrary well-formed program structures. Once the identities are known, the contents of the entities can be referred to by some keys, instead of the contents themselves. This enables many chances to reuse program components. It plays a role in building modularized programs, just like variables to a high-level programming language (which enable the reusing of values or states by name; although the real identity problem is solved by having lvalues and mutable states, rather than just variables). For this reason, totally missing of such guarantees is bad, apparently.
A case study
For binary program modules, it is ideal to have any entities in the universe of libraries by some identifiers (symbols) from known location (library paths).
When the an identifier and a location form a key, it can replace the contents it referred to. Then it is easily reused in different passes during translation. Linking is the stage to put them together. The property of the identifier with some definitely correct way to find the content (definition of entities implied in the language) is called the linkage. This is exactly the separation of translation and linking (and Rust follows this practice).
Coherence in this context is the binary compatibility among any exposed ("public") entities binary modules, which providing a set of stable API. Given that there can be infinite entities, there must be some rules to ensure the coherence. This is the ABI (corresponding to the specific API, derived form both the API and ABI specifications). Basically, the violation of compatibility imposed by ABI requirements are (hopefully) all checked in the linking stage, during the symbol resolution by the linker. There can be other variants like delaying the symbol resolution to loading by a dynamic loader.
However, it is generally difficult to lift the check before the linking stage. It cannot be finished during compiling due to lacking of non-local (to the compilation unit being compiled) information.
Rust's coherence is essentially a weaker form of this. It has some more assumptions and it can be finished during compiling. But many problems remain. See below.
When
Do we really want such properties to be enforced by the compiler checks (if possible)?
In practice such (informal) coherence implies a very strong assumption that all Rust types are in the same type universe, but this is virtually impossible. For example, any update of the language version which modifies the universe cannot be coherent with each other. It split the worlds, daily and nightly. (Now just forget this, because there are more serious problems.)
This falls into somewhat like the dilemma of binary compatibility: everyone want it for free because it sounds nice and lacking of it seems obviously horrible, but for many reasons, nobody in real systems can get it even close to right, in the language design.
Because there is simply lacking of sufficient information for such properties provable by the language rules, in most languages not bothering to dig the implementation details too much.
This is not a problem of choice. Force it too early leads to less ideal quality, either less powerful guarantees, or limitations on program structure.
How (and how bad)
Here Rust weakens the coherence properties on Rust types which can have impls. This is weakening because the entities set is restricted in the Rust type universe (albeit still infinite, so there requiring rules a priori).
As a nominal typing language, Rust has type identity determined by name in most contexts. This means users can have source code with same name but different definitions quite easily, so checks in the same TU (translation unit; in Rust, a crate) is a must. This process also does the check to handle the error conditions like redefinition. But this is not applicable across different TUs. From the toolchain's view, it need to share metadata from different TUs, so this should be done by the linker, but not the compiler. The compiler has no sufficient information to do this correct. Only with sufficient strict rules like the so called orphan rules here, the compiler can prove that there is one-to-one mapping from name (path) to definition of type, but this is merely a special case. This is too restrictive with lots of false negative results. It prevents code otherwise consistent, and finally it harms the extensibility.
The direct problem
Even the ability to check such properties by compilers are needed, the current rules are obviously flawed, besides the well-known inconvenience among 3 crates. It's about providing implementation by the user of a library. Worse, the "newtype" workaround does not work for this case.
The current way of the orphan rules enforces that only the current crate able to be the owner of some definitions keyed by specific signatures. This is unfortunately, where the signature is not formed by a single path.
For instance, T<U> can have 2 names T and U which are from different crates. The definition of local type drops U totally. When T is foreign, the check fails because T<U> is also considered non-local.
Now consider, if T<U> is non-local, where is the place that it is local? When U is user-provided and T is in a library, the author of T cannot assume U is enumerable in general, so definitions appertain to T<U> cannot live in the library crate. And the rules reject them to be in the place providing U. There is no way to provide it directly, besides the so-called "newtype" workaround, but such workaround may stop working when the type identity of T<U> is publicly acknowledged (say, in some language binding boundaries).
Hence, there is no extensibility to make T<U> coexist unless T and U are merged to a single crate. This is anti-modular-ism, and it defeats the purpose to having different crates. Totally absurd.
Alternatives
What about other languages?
A comparable instance is C++. The coherence of entity definitions are enforced normatively by ODR (one definition rule) in ISO C++. The requirements of checks vary. The intra-TU checks of duplication of definitions are required; the inter-TU checks are not mandated and it can be IFNDR (ill-formed but no diagnostics required).
It is acknowledged that Rust's coherent rules are analogue of C++'s ODR. (It is interesting that Rust crates are DAG nodes, but this has little to do with the topic here. It is incorrect to say DAG is a must to the static checks of coherence; a graph traverse can also achieve that... with a lot more complexity, though.)
Here the differences between the design are obvious again: C++ implementations have morally right (just working) checks during linking; while Rust checks it during compiling.
It's nothing wrong to make the check earlier, because detecting a mistake earlier than later is nice in general, except that it is not expected as a mistake.
Another concern is that under-specification (like in ISO C++) can be complemented with extensions by implementations. Although IFNDR (essentially UB during translation-time) voids any guarantees by the language specification, the implementations can provide more rules. Note ISO C++ only requires a few definitions like operator new being able to be redefined, and it is exactly implemented under the same spirit to resolve ODR violations, notably, weak symbols in ELF targets. Taking such possibility in mind, treating ODR violation too early is actually a symptom of overly designing.
As a result, the problem of extensibility above does not happen in C++. That does not mean the arbitrary unchecked definitions are allowed, even before the linking. ISO C++ has stronger (more explicit) rules in the standard library to disallow randomly additions to the std namespace and specializations of arbitrary library-provided templates. This captures more or less the legitimate intent to prevent unauthorized modifications on already an existing "closed" design of a library, albeit, totally unchecked; and these rules have more coverage: not only the potential collisions of (type) definitions, but all kinds of unwanted modifications and customization, if actually exists, to the library interface. And more importantly, it is not a mandated check enforced by core language rules; it is library-scoped, instead. This is far more flexible. If the author of some library have different thoughts to support a distinct usage of the library than standard library, he/she can use some different strategies. ODR from the core language is still the baseline, though.
In particular, note the ISO C++ rules do the right thing here: it typically allows templates being specialized for template instances depending on at least one user-provided type. This is directly against the "local type" definition in the orphan rules. As of Rust, besides the inflexibility due to the mixture of rules for different purposes... Errr, Rust still does not have the stabilized specialization even in 2024...
And what about other libraries?
It might be worth noting that serde has a whole set of sophisticated mechanism to interact with types from different crates. This is essentially bypassing the nominal checks for structural type equivalence, in the same spirit of usual rules of foreign ABI and interops, except there is also some additional compile-time checks fabricated by serde itself to ensure the static type safety (in a way more like ABI rules than Rust ones).
Nonetheless, it is also worth noting, the serde is special enough because building a framework of serialization/deserialization representations is its core work, so it exactly has the legitimate rights to "abuse" the assumption to work around the limitation of the language rules. This does not mean any normal library should do the same, i.e. it should not be idiomatic.
Conclusions
It might be disputable and inappropriate to simply conclude that the orphan rules are ill-designed; they are likely better than nothing. Whatever, there are many unaddressed problems making troubles.
At least, lacking of the support of user-provided parameterized types is an obvious hole. This should be fixed.
Some problems have raised naturally about the decision to have the rules in the core language.
(See also a somewhat related issue #22.)
What
The initial problem is that, the so-called [in]coherence is never well-defined besides the language rules themselves, if the non-existence of coherence has ever been a problem.
This is a chicken-and-egg problem. It is doubtful whether a formal definition can be precisely established in this context. Probably some informal intents still make sense. However, I see few people respect the background problems sufficiently; many users insists on the necessity of the so-called coherence, just because it sounds goodTM plus totally lacking of such properties is obviously horrible, intuitively (so we better have it).
Informally, the so-called coherence is generally (not specifically to Rust) a group of properties about the identity of entities introduced in arbitrary well-formed program structures. Once the identities are known, the contents of the entities can be referred to by some keys, instead of the contents themselves. This enables many chances to reuse program components. It plays a role in building modularized programs, just like variables to a high-level programming language (which enable the reusing of values or states by name; although the real identity problem is solved by having lvalues and mutable states, rather than just variables). For this reason, totally missing of such guarantees is bad, apparently.
A case study
For binary program modules, it is ideal to have any entities in the universe of libraries by some identifiers (symbols) from known location (library paths).
When the an identifier and a location form a key, it can replace the contents it referred to. Then it is easily reused in different passes during translation. Linking is the stage to put them together. The property of the identifier with some definitely correct way to find the content (definition of entities implied in the language) is called the linkage. This is exactly the separation of translation and linking (and Rust follows this practice).
Coherence in this context is the binary compatibility among any exposed ("public") entities binary modules, which providing a set of stable API. Given that there can be infinite entities, there must be some rules to ensure the coherence. This is the ABI (corresponding to the specific API, derived form both the API and ABI specifications). Basically, the violation of compatibility imposed by ABI requirements are (hopefully) all checked in the linking stage, during the symbol resolution by the linker. There can be other variants like delaying the symbol resolution to loading by a dynamic loader.
However, it is generally difficult to lift the check before the linking stage. It cannot be finished during compiling due to lacking of non-local (to the compilation unit being compiled) information.
Rust's coherence is essentially a weaker form of this. It has some more assumptions and it can be finished during compiling. But many problems remain. See below.
When
Do we really want such properties to be enforced by the compiler checks (if possible)?
In practice such (informal) coherence implies a very strong assumption that all Rust types are in the same type universe, but this is virtually impossible. For example, any update of the language version which modifies the universe cannot be coherent with each other. It split the worlds, daily and nightly. (Now just forget this, because there are more serious problems.)
This falls into somewhat like the dilemma of binary compatibility: everyone want it for free because it sounds nice and lacking of it seems obviously horrible, but for many reasons, nobody in real systems can get it even close to right, in the language design.
Because there is simply lacking of sufficient information for such properties provable by the language rules, in most languages not bothering to dig the implementation details too much.
This is not a problem of choice. Force it too early leads to less ideal quality, either less powerful guarantees, or limitations on program structure.
How (and how bad)
Here Rust weakens the coherence properties on Rust types which can have
impls. This is weakening because the entities set is restricted in the Rust type universe (albeit still infinite, so there requiring rules a priori).As a nominal typing language, Rust has type identity determined by name in most contexts. This means users can have source code with same name but different definitions quite easily, so checks in the same TU (translation unit; in Rust, a crate) is a must. This process also does the check to handle the error conditions like redefinition. But this is not applicable across different TUs. From the toolchain's view, it need to share metadata from different TUs, so this should be done by the linker, but not the compiler. The compiler has no sufficient information to do this correct. Only with sufficient strict rules like the so called orphan rules here, the compiler can prove that there is one-to-one mapping from name (path) to definition of type, but this is merely a special case. This is too restrictive with lots of false negative results. It prevents code otherwise consistent, and finally it harms the extensibility.
The direct problem
Even the ability to check such properties by compilers are needed, the current rules are obviously flawed, besides the well-known inconvenience among 3 crates. It's about providing implementation by the user of a library. Worse, the "newtype" workaround does not work for this case.
The current way of the orphan rules enforces that only the current crate able to be the owner of some definitions keyed by specific signatures. This is unfortunately, where the signature is not formed by a single path.
For instance,
T<U>can have 2 namesTandUwhich are from different crates. The definition of local type dropsUtotally. WhenTis foreign, the check fails becauseT<U>is also considered non-local.Now consider, if
T<U>is non-local, where is the place that it is local? WhenUis user-provided andTis in a library, the author ofTcannot assumeUis enumerable in general, so definitions appertain toT<U>cannot live in the library crate. And the rules reject them to be in the place providingU. There is no way to provide it directly, besides the so-called "newtype" workaround, but such workaround may stop working when the type identity ofT<U>is publicly acknowledged (say, in some language binding boundaries).Hence, there is no extensibility to make
T<U>coexist unlessTandUare merged to a single crate. This is anti-modular-ism, and it defeats the purpose to having different crates. Totally absurd.Alternatives
What about other languages?
A comparable instance is C++. The coherence of entity definitions are enforced normatively by ODR (one definition rule) in ISO C++. The requirements of checks vary. The intra-TU checks of duplication of definitions are required; the inter-TU checks are not mandated and it can be IFNDR (ill-formed but no diagnostics required).
It is acknowledged that Rust's coherent rules are analogue of C++'s ODR. (It is interesting that Rust crates are DAG nodes, but this has little to do with the topic here. It is incorrect to say DAG is a must to the static checks of coherence; a graph traverse can also achieve that... with a lot more complexity, though.)
Here the differences between the design are obvious again: C++ implementations have morally right (just working) checks during linking; while Rust checks it during compiling.
It's nothing wrong to make the check earlier, because detecting a mistake earlier than later is nice in general, except that it is not expected as a mistake.
Another concern is that under-specification (like in ISO C++) can be complemented with extensions by implementations. Although IFNDR (essentially UB during translation-time) voids any guarantees by the language specification, the implementations can provide more rules. Note ISO C++ only requires a few definitions like
operator newbeing able to be redefined, and it is exactly implemented under the same spirit to resolve ODR violations, notably, weak symbols in ELF targets. Taking such possibility in mind, treating ODR violation too early is actually a symptom of overly designing.As a result, the problem of extensibility above does not happen in C++. That does not mean the arbitrary unchecked definitions are allowed, even before the linking. ISO C++ has stronger (more explicit) rules in the standard library to disallow randomly additions to the
stdnamespace and specializations of arbitrary library-provided templates. This captures more or less the legitimate intent to prevent unauthorized modifications on already an existing "closed" design of a library, albeit, totally unchecked; and these rules have more coverage: not only the potential collisions of (type) definitions, but all kinds of unwanted modifications and customization, if actually exists, to the library interface. And more importantly, it is not a mandated check enforced by core language rules; it is library-scoped, instead. This is far more flexible. If the author of some library have different thoughts to support a distinct usage of the library than standard library, he/she can use some different strategies. ODR from the core language is still the baseline, though.In particular, note the ISO C++ rules do the right thing here: it typically allows templates being specialized for template instances depending on at least one user-provided type. This is directly against the "local type" definition in the orphan rules. As of Rust, besides the inflexibility due to the mixture of rules for different purposes... Errr, Rust still does not have the stabilized specialization even in 2024...
And what about other libraries?
It might be worth noting that
serdehas a whole set of sophisticated mechanism to interact with types from different crates. This is essentially bypassing the nominal checks for structural type equivalence, in the same spirit of usual rules of foreign ABI and interops, except there is also some additional compile-time checks fabricated byserdeitself to ensure the static type safety (in a way more like ABI rules than Rust ones).Nonetheless, it is also worth noting, the
serdeis special enough because building a framework of serialization/deserialization representations is its core work, so it exactly has the legitimate rights to "abuse" the assumption to work around the limitation of the language rules. This does not mean any normal library should do the same, i.e. it should not be idiomatic.Conclusions
It might be disputable and inappropriate to simply conclude that the orphan rules are ill-designed; they are likely better than nothing. Whatever, there are many unaddressed problems making troubles.
At least, lacking of the support of user-provided parameterized types is an obvious hole. This should be fixed.