Deep Merging Utility Types

[SephReed]

edit: changed the word "union" to "merge" to avoid confusion

Suggestion

🔍 Search Terms

"deep union" "sub property merge" "deep merge"

✅ Viability Checklist

My suggestion meets these guidelines:

• [x] This wouldn't be a breaking change in existing TypeScript/JavaScript code
• [x] This wouldn't change the runtime behavior of existing JavaScript code
• [x] This could be implemented without emitting different JS based on the types of the expressions
• [x] This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, new syntax sugar for JS, etc.)
• [x] This feature would agree with the rest of TypeScript's Design Goals.

⭐ Suggestion

Deep type merges with objects that have enums or functions props are not fully possible in Typescript, nor are their "best" implementations very performant or transparent.

It would be nice to have a generic type for deep merges that is:

1. terse
2. extends the base generics -- as in DeepMerge<A, B> can take everything you'd throw at A or B
3. holds optional values -- optional and required values are logically set
4. is performant

5. debuggable -- it's easy to tell what's going on when you merge a few types

   Here are few examples of how to do them now, and how they hold up

Eg 1: The one I found

export type SubpropertyMerge<T> = (
  T extends (...args: infer A) => infer R ? (
    (...args: SubpropertyMerge<A>) => SubpropertyMerge<R>
  ): (
    T extends object ? { [K in keyof T]: SubpropertyMerge<T[K]> } : T
  )
);

• terse: while not extremely terse, it's decent
• extends generics: big headaches the moment you start including functions. also string | { someProp: any } will break the thing by unioning empty objects (I got ~100 {} | {} | {} from this).
• holds optionals: seems to work just fine
• performant: not great.. I get noticeable type-check lag once it's a few layers deep (2017 MBP)
• debuggable: not really, the types come out terrible

Note: if you make the output type for functions (...args: A) => R (instead of merging sub-properties), it seems to handle functions at least sort of decently.

Eg 2: The best I could manage

export type DeepMerge<T1, T2> = (
  T1 extends object ? (
    T2 extends object ? (
      MergeToOne<(
        { [K in (keyof T2 & keyof T1 & RequiredKeys<T1 | T2>)]: DeepMerge<T1[K], T2[K]> }
        & { [K in (keyof T2 & keyof T1 & OptionalKeys<T1 | T2>)]?: DeepMerge<T1[K], T2[K]> }

        & { [K in Exclude<RequiredKeys<T1>, keyof T2>]: T1[K] }
        & { [K in Exclude<OptionalKeys<T1>, keyof T2>]?: T1[K] }

        & { [K in Exclude<RequiredKeys<T2>, keyof T1>]: T2[K] }
        & { [K in Exclude<OptionalKeys<T2>, keyof T1>]?: T2[K] }
      )>
    ) : (
      T1 extends object ? T2 : T1 | T2
    )
  ) : (
    T2 extends object ? T1 : T1 | T2
  )
);

// so you don't get "T & {} & {}"
// also assumes that "undefined" is only ever a value included by optional params... I couldn't find a way around this
type MergeToOne<T> = (
  T extends object ? { [K in keyof T]: (
    K extends RequiredKeys<T> ? Exclude<T[K], undefined> : T[K]
  )} : never
)

type RequiredKeys<T> = { [K in keyof T]-?: {} extends Pick<T, K> ? never : K }[keyof T];
type OptionalKeys<T> = { [K in keyof T]-?: {} extends Pick<T, K> ? K : never }[keyof T];

• terse: not at all
• extends generics: seems to work pretty well, as long as "undefined" is not a possible value for anything (use null instead)
• holds optionals: a lot of optionals include "undefined" as a value type... there's no way to tell if the optional did it, or if the user specified that undefined was a valid value
• performant: I've got quite a few layers of these, and it takes ~20 second for type checking to catch up with any change I make. It's a bummer: I finally got a type to describe my project (~40hrs of creating the perfect type), and now it's really really slow.
• debuggable: surprisingly, the output types are extremely readable from this one

Note: this one doesn't even attempt to merge functions. Also, the T1 extends object ? T2 : T1 | T2 that comes right after having just checked if T1 extends object ? is necessary. I don't know why, and the amount of difficulty it takes to grasp this system is yet another reason I think it would be nice to just have a fully tested Utility type instead.

📃 Motivating Example

type A = { bar: { qux?: 1 }};
type B = { bar: { qux: 2 }, baz: null}

type Foo = DeepMerge<A, B>;

const foo: Foo = {
   bar: { qux: 1 }, // qux is non-optional
   baz: null
};  
foo.bar.qux = 2;
foo.bar.qux = 3; // type error - must be 1 | 2

💻 Use Cases

Almost any time a user uses a normal merge A & B on objects, they're probably expecting a deep merge. Eg:

Honestly, given how much more useful a deep merge is than a shallow one, it almost seems worthy of being && rather than a utility type. Eg. A && B

⁉️ Complications

There's a lot of different ways of merging two functions

1. (X) => Y | (A) => B
2. or (X | A) => Y | B
3. or (DeepMerge<X, A>) => DeepMerge<Y, B>

For the sake of suggestion, I'd probably go with # 3 above because it encompasses # 1 and # 2 a lot better than either can encompass # 3. You can give # 3 the same arguments as # 1 or # 2 and the worst it will do is demand some extra properties. The other way around, and it won't have any typings and will throw errors for the excess values. It's probably better to demand something useless with proper type hints, than deny something necessary with no type hints.

[fatcerberus]

extends the base generics -- as in DeepUnion<A, B> extends A and extends B is true

This sounds more like an intersection than a union. Union types are never directly assignable to any of their constituent types (without narrowing them first).

[SephReed]

I'm not positive on the semantics here. There are some code examples above which show what I'm talking about. Please feel free to change any vocabulary I may have misused or misunderstood.

Also, I never said "directly assignable," I'm speaking more of generics X extends Y stuff... which may be impossible, but this is only one of many things such a utility is necessary for.

As it stands, my project can not type check in less than 15 seconds because of how gnarly the home-brewed deep merge is.

[SephReed]

Hmmm.... I may have found a solution for now:

export type SubpropertyMerge<T> = (
  T extends (...args: infer A) => infer R ? (
    (...args: A) =>  R
  ): (
    T extends object ? (
      T extends string | number | ((...args: any) => any) | symbol | boolean ? T
      : { [K in keyof T]: SubpropertyMerge<T[K]> }
    ) : T
  )
);

By adding the "catch if extending object + string" case, things appear to be working (and type checking within a few seconds).

Is there a way to specifically state T extends object and nothing else? Assuming this FR isn't going anywhere, it'd be nice to at least share the best possible vanilla form of this functionality.

[fatcerberus]

A & B is an intersection, not a union. As for assignability: because of the way the TS structural type system works, A extends B means exactly “A is assignable to B”.

[SephReed]

https://www.typescriptlang.org/play?ssl=3&ssc=18&pln=3&pc=24#code/FDAuE8AcFMAICECGBnOBeWBvWAPAXLAHYCuAtgEbQBOsAviAGbGEDGoAlgPaGyiIDW0ZElQAeACqxoOUNEIATZAhTQAfAApyBcQEos9MAKEjoozPlgAmWhvMFrOgNxA

type Base = { x: number }

function takesBase<T extends Base>(b: T) {}

takesBase<{x: 2}>({x: 2});

Something like this, though it's not the major facet here. Please see the "Use Cases" section. It makes it pretty clear what's confusing.

[MartinJohns]

Please see the "Use Cases" section. It makes it pretty clear what's confusing.

The example in your "Use Cases" section is honestly very confusing. You say when people write A & B they expect a union, but why would they expect a union when they use the intersection operator? In the screenshot you can see exactly what I would expect: tt.foo being never, because the intersection of A and B results in an impossible type.

[SephReed]

Hmmm... I'm going to try to explain again:

1. Semantics:

I don't think what I'm talking about is either a union or an intersection. Perhaps I'll call it deep merge. I'm going to change the post to remove any reference to unions.

2. The use case:

So for something like the following example, things work great:

type Bear = { hands: { bigAndScary: true }}
type Frog = { hands: { sticky: true }}
const bearFrog: Bear & Frog = null as any;
bearFrog.hands.bigAndScary; // true
bearFrog.hands.sticky; // true

But for something like the following example, things fall apart:

type Jukebox = { accepts: "quarters";  playsMusic: true; }
type Atm = { accepts: "cards"; hasTouchScreen: true; }

const strangeMachine: Jukebox & Atm = null as any;

typeof strangeMachine; // never

const strangeMachine2: Jukebox | Atm = null as any;
strangeMachine2.accepts; // yay! it works
strangeMachine2.playsMusic; // darn... not a prop
strangeMachine2.hasTouchScreen; // not a prop

Rather than strangeMachine being a machine that does all the things of both its component machines, it does none of them... or just one part of them. I both understand the logic here, and feel it as intuitive. If someone said "give me something thats both a jukebox and atm", I wouldn't think they wanted either of the outcomes of union or intersection. I'd think they wanted whichever one was possible on a property to property basis. Like this:

type StrangeMachine = {
  accepts: "quarters" | "cards";       // union here
  playsMusic: true;                            // more of an intersection here
  hasTouchScreen: true;                   // and here
}

3. Extensions

I was wrong to say StrangeMachine should extends Jukebox and ATM. It's more that -- with some effort -- you can set things up so that your "Deep Merges" will extend some super generic base, to further define it. Kind of like carving marble. A user can start off with something super open ended, and then deep merge to "narrow" the fields... then deep merge more to open them a bit.

type MarbleStatue = { focus: string; }
type GreekStatue = { focus: "lifelike"; polished: true; }
type AbstractStatue = { focus: "geometric"; verySubjectiveMeaning: true; }

function statueFn<T extends MarbleStatue>(statue: T) { return statue; }

const fusionStatue: DeepMerge<GreekStatue, AbstractStatue> = {
  focus: "geometric",  // choices "lifelike" | "geometric"
  polished: true,
  verySubjectiveMeaning: true,
}

// can still be passed to this base function
statueFn(fusionStatue); 

This is actually a fairly decent example of what I need deep merging for: crafting an interface from a collection of smaller ones. I'm working with a plugin system where entities can have functionality mixed and matched... class extension does not work here, so deep merging configs is the solution.

4. Confusions

Okay, that's a lot to write. I hope this clears up some confusion. If not, maybe we'll try again? Thank you for your patience.

[RyanCavanaugh]

Types like these have a ton of what I call "policy" decisions:

• Does merging an optional property with a required one make a new optional property, or a required one?
• How does merging work when both sides contribute a union of a primitive and object type?
• How do function types merge (overloading, or by combining parameters by some rule?)
• How should index signatures merge?

It seems like the ruleset you have here says that primitives should union and object types should merge, which is definitely neither a union nor an intersection. It's a domain-specific merging rule. Writing these in userspace is totally appropriate, because you have to make a ton of decisions about how this merging works. It's not tractable for us to provide intrinsic type operators to cover the dozens of ways you could write this "merge" type.

[SephReed]

Welp, I can already tell this proposal is dead.

Everyone knows that perfect is the enemy of good. Merging interfaces of a depth > 1 can not be done perfectly for every possible use case. It can be done well enough for people to get on with their day though.

So the answers to your question are pretty easy from a "my job it to write code and wrestling type systems wastes my time" perspective:

• it is better to require an optional variable, than typelessly deny a required one
  • false requirement?: throw a 0 as any in there and move on (or if fancy, use a Symbol called "Opt" or something)
  • typelessly denied requirement?: you've got a lot of type wrestling to do. @ts-ignore is not a good option

1. optional or required?: flip a coin. There's pro's and con's to both, and it's both easy to write run-time code to assert and easy to throw a Opt as any there. Either way, I'll be able to keep working.
2. merging or unioning?: merge. false requirements are easy to deal with, denied requirements ruin a day
3. functions?: merge args and return would fit the "false requirements > typelessly denied ones` rule. A second type that unions functions instead of merging them would probably make people happy though.
4. not sure what you mean by index signatures, but the golden rule probably still applies.

---

But, like I said above, perfect is the enemy of good. Trying to merge a couple interfaces just to get never is a bit of a day stopper. Literally anything is better than that.

---

I know when a proposal is dead, and this one is very very dead. I've seen far better proposals die from perfectionist critique.

For anyone who comes across this trying to combine interfaces of a depth greater than 1, here's the best I've found yet:

export type SubpropertyMerge<T> = (
  T extends object ? (
    T extends string | number | ((...args: any) => any) | symbol | boolean ? T
    : { [K in keyof T]: SubpropertyMerge<T[K]> }
  ) : T
);

[fatcerberus]

As far as I can tell it’s not about “good” vs. “perfect”, it’s about having different but equally valid ways to do it, and it will likely be impossible to reach consensus on what is the best approach. That goes double if we’re talking about a built-in operator as opposed to a utility type.

As for getting never: that’s because you’re trying to use the “intersection” operator as a “merge” operator, when that’s not actually what it is. A & B means precisely “give me a type that’s assignable to both A and B”. Very simple and mathematical. Sometimes that’s impossible, so you get never, meaning there’s no value that satisfies that requirement.

[SephReed]

A & B means precisely “give me a type that’s assignable to both A and B”. Very simple and mathematical.

I've given you guys a lot of understanding here, and I've yet to see it returned.

It is very logical and simple. Logic and intuition don't always align. And -- as it turns out -- the intuitive answer to "what happens when you combine an ATM and a Jukebox" is not "never".

[MartinJohns]

I've given you guys a lot of understanding here, and I've yet to see it returned.

I'd say you're understood by now.

the intuitive answer to "what happens when you combine an ATM and a Jukebox" is not "never".

The intuitive answer to "I want a single screen machine being both at once an ATM with an OLED screen and a Jukebox with an LCD screen" is "you can't have that, it doesn't make sense and can't exist".

You're just leaving out too many details.

[SephReed]

I'd say you're understood by now.

Okay, then what would I give you if you asked me for a combination ATM/ Jukebox?

[tjjfvi]

const strangeMachine: DeepMerge<Jukebox, Atm> =

Can you give an example of a valid value for this?

[SephReed]

type Jukebox = { accepts: "quarters";  playsMusic: true; }
type Atm = { accepts: "cards"; hasTouchScreen: true; }

const strangeMachine: DeepMerge<Jukebox & Atm> = {
  accepts: Math.random() < 0.5 ? "quarters" : "cards",     
  playsMusic: true,               
  hasTouchScreen: true,               
}

---

An example of what I needed this for (and have since figured out) is combining configs.

I have a system in which "augments" are chained together to create a "factory." Depending on the augments, the factory will require more complex config (with more options, etc), and will produce more complex function sets. Both the configs and function sets are a product of a deep merge right now.

It's a way of getting around class inheritance, and basically the same thing as decorators but you can actually modify the type of the constructor arguments.

It's working like a charm right now, but I had to hack a bit around typescript a bit because not having a utility type made the whole thing recursively complex... it was taking 3-5 seconds for the language server to respond to any changes.