Handling of indirect access of `extern` types

[jonmeow]

Summary of issue:

How should complete/incomplete types work for indirect use mixing extern and non-extern versions of the same type?

Related discussion on #3980

This has some relationship with the has_extern question on #3986, but I think that's focused on syntax and this is behavior.

Details:

Suppose you have the code:

package P library "extern";
extern class C;

package P library "extern_fn";
import library "extern";
fn F1() -> C*;

package P library "owner";
import library "extern";
[has_extern] class C { x: i32 }

package P library "owner_fn";
import library "owner";
fn F2() -> C;

library "use";
import P library "extern_fn";
import P library "owner_fn";

fn Use() {
  var a: i32 = P.F1()->x;
  var b: i32 = P.F2().x;
}

What is the expected result here?

• Is C complete for F1()->x?
• Would it be complete if the calls are reversed?
• Does the "use" library need to import "owner" directly in order to get a complete type?

I think there are three main options:

1. The order of calling functions matters, under the information accumulation principle.

   • The code as written is invalid, but reversing F2() and F1() calls results in valid code.
   • Calling F2 results in its type information being accumulated, making C complete.

2. Whenever a library has an exported entity which uses a has_extern type, it adds that has_extern type to a list of types which must be evaluated on import. This is not transitive unless re-used.

   • The code as written is valid, and would remain valid without the F2() call, although additionally removing import P library "owner_fn"; would break the code.

   • There's a simpler option here where all imported has_extern can be treated this way, but the restriction to exported entities may simplify matters. Although in turn, it may become really complex for class members, or other non-namespace-scoped entities (which should have the same consequence because of how they would be accessed, but only namespace-scoped entities are directly exported, implying a recursive search for has_extern uses).

3. If a has_extern type is indirectly returned (and not directly imported) the type is always incomplete.

   • The code as written is invalid and requires import P library "owner".
   • Definitions of types are transitively visible (even if not explicitly re-exported) if they are not has_extern
   • Definitions of has_extern types are not transitively visible, and those types are instead incomplete if their owning library (or a library that export name;s the type, or maybe export name;s something involving the type?) is not directly imported.
     • One small edit to zygoloid's wording here: this means that has_extern exports are also treated differently from other exports. In the example code, C is never named, but needs to be loaded as a complete type regardless, meaning all has_extern types that are directly imported are completed (similar to option 2).

Note export functionality will likely interact in interesting ways with the latter two options.

[jonmeow]

Note, as a corollary (also brought up by zygoloid on #3980, but I think I should be explicit here) -- it might be nice to also decide, in a similar situation for a non-extern type (a function returning the type when it's not directly imported), if the behavior should be equivalent. Types with extern uses might reasonably diverge, although it does create two behavior paths.

Right now, I think prior discussion had leaned towards the concept that a non-extern type provided as an indirect return -- not directly imported -- remains complete/usable (use by name still requires an import). IIRC rationale was that it's more convenient not to have to explicitly import (or re-export) return types; it allows APIs to change the return type without requiring callers to update imports in order to use the return type. I don't think we have that formally written down though, unless I'm forgetting something.

[chandlerc]

Somewhat referenced in a summary idea on #3980, but relaying the specific aspect here...

I lean towards meaning (3), and to be explicit:

• Is C complete for F1()->x?

No.

• Would it be complete if the calls are reversed?

No.

• Does the "use" library need to import "owner" directly in order to get a complete type?

Yes.

And to combine that with the corollary for clarity:

a non-extern type provided as an indirect return -- not directly imported -- remains complete/usable (use by name still requires an import).

Yes. For non-extern types, you must Import What You Spell, or more technically in Carbon, what you want to name. But the type and its definition are transitively provided and available. And it specifically this that is the fundamental property of an extern type -- it's definition is not transitively available, and it most be imported directly everywhere it is used.

[chandlerc]

I think we're good here from the leads with (3) as mentioned.