Proposal: support pathof along with current keyof

[agalazis]

Problem - What are users having difficulty with?

While using popular libraries for their day to day development needs, developers can find them disjoint from typesafe philosophy even if they do have the needed typings. This is true for frequently used libraries such as lodash and immutablejs that access or set properties via paths. It is up to the developers to to preserve typesafety by expressing complex types using the amazing power that typescript gives them. The only obstacle is the absence of a way to express the type that represnts legit object paths for a specific type.

Current state

We can currently do this only for shallow objects where paths can be simply expressed as the keys of a specific type.

Example Issue

In an effort to play with stricter typings for immutablejs map I created this tiny project : https://github.com/agalazis/typed-map/ (as a side note my personal view is that immutable js map is not a conventional map that should be represented with map<keyType, valueType> since it represents a map that matches a specific type rather than just a key,value data structure as demonstarated in src/examples)

The type I created was just this (only playing with get and set):

export type TypedMap<T> = {
  get: <K extends keyof T >(k:K) => T[K];
  set: <K extends keyof T >(k:K, v:T[K]) => TypedMap<T>;
}

Simple enough, leveraging all the expressiveness of typescript.

Example usage of the proposed solution:

If I could replace keyof with pathof in order to express possible path strings and also used T[P] as the path type I would be able to completely cover this use-case :

export type TypedMap<T> = {
  get: <P extends pathof T >(p:P) => T[P];
  set: <P extends pathof T>(p:P, v:T[P]) => TypedMap<T>;
}

Possible usage in the following libraries:

• lodash
• immutable
• ramda
• anything built on top of them

Why bother since this does not involve facilitating any feature of ES standard?(and is somewhat an unconventional feature)

1. We will be able to perform type-safe updates using existing js immutability libraries similar to what you can achieve in other languages (eg. skala lense).
2. This feature complies with typescript goals:
   • Statically identify constructs that are likely to be errors.
   • Provide a structuring mechanism for larger pieces of code (the above-mentioned libraries are used every day in big projects imagine the chaos that can be created. Despite exhaustive unit testing, intellisense is a must ;) ).
   • Strike a balance between correctness and productivity.
3. This feature improves compile time checks and sticks to the mindset of not having to deal with any runtime functionality
4. There are no side effects in the generated js code

Alternatives

While digging further into the issue an alternative solution is to be able to spread keyof recursively. This will allow us to be creative and build our own solution as per: https://github.com/Microsoft/TypeScript/issues/20423#issuecomment-349776005 The drawbacks of the alternative solution if doable are:

• performance
• lack of cyclic reference support

Implementation Suggestions

The optimal (if implemented in the language) would be to not compute the full nested object paths but only the paths used via pathof ie when declaring something as path of x just accept it as pathof x then when a value is assigned just validate it is pathOf x( the compiler could also have some sort of caching so that it doesn't recalculate paths). This will also solve the issue with cyclic references since paths will be finite anw.

[noppa]

So, if I understand correctly, the idea is that you could have something like

var m: TypedMap<{ a: { b: number } }>;
var m2 = m.set('a.b', 42);

where 'a.b' would be of type pathof { a: { b: number } }.

You can already have type safety for a similar function

m.set('a', 'b', 42);

with the following type definition for TypedMap:

interface TypedMap<T> {
  get<K extends keyof T>(k: K): T[K];
  get<K extends keyof T, K2 extends keyof T[K]>(k: K, k2: K2): T[K][K2];
  get<K extends keyof T, K2 extends keyof T[K], K3 extends keyof T[K][K2]>(k: K, k2: K2, k3: K3): T[K][K2][K3];

  set<K extends keyof T>(k:K, v:T[K]): TypedMap<T>;
  set<K extends keyof T, K2 extends keyof T[K]>(k:K, k2: K2, v:T[K][K2]): TypedMap<T>;
  set<K extends keyof T, K2 extends keyof T[K], K3 extends keyof T[K][K2]>(k:K, k2: K2, v:T[K][K2][K3]): TypedMap<T>;
}

Now, this approach does have a few drawbacks, of course; the API isn't quite as neat and the supported depth of the objects is limited by the number of overloaded function definitions you are willing to write.

Still, most of the use cases you mention could be achieved with the current state of the type system.

Playground link.

[dmichon-msft]

One problem is that the following is completely legal:

interface A {
    a: boolean;
    b: 2;
    'a.b': string;
    '"a.b"': number;
}

type T1 = A['a']; // boolean
type T2 = A['a.b']; // string
type T3 = A["'a.b'"]; // string[]
type T4 = A['"a.b"']; // number

In fact, any path string you can possibly come up with is a fully legal keyof T for some T, so you'll run into collisions between the two.

[agalazis]

@noppa Thank you for contributing your thoughts yes the above is another way to approach this issue but as you mentioned, it has the depth limitation but also a productivity issue(if we manually have to do all the overloading or rewrite typings for every type since depth is variable). TS goals: Strike a balance between correctness and productivity You just demonstrated another way that cannot be used for shipping plug and play typings for the mentioned use case/libraries and confirmed my suspicion that this is not currently doable( a strong indication is also that none of the above-mentioned libraries supports such strict checks). On the other hand, based on the comments, I started looking into a way of expressing the path as an array since path as array of keys is something commonly used:

type PathSetter<T, K extends keyof T ,P=K,M=TypedMap<T>> =  ( p:P, t: T[keyof T]) => M | PathSetter<T[K], keyof T[K], [P, ...keyof T[K]], M>
export type TypedMap<T> = {
  get: <K extends keyof T >(k:K) => T[K];
  set: <K extends keyof T >(k:K, v:T[K]) => TypedMap<T>;
  setIn: PathedSetter<T, keyof T>;

}

which won't work since I can't use spread operator on keys

[noppa]

My example would also work with tuple types, which might be closer to what you are looking for

interface TypedMap<T> {
  set<K extends keyof T>(k:K, v:T[K]): TypedMap<T>;
  set<K extends keyof T, K2 extends keyof T[K]>(k: [K, K2], v:T[K][K2]): TypedMap<T>;
  set<K extends keyof T, K2 extends keyof T[K], K3 extends keyof T[K][K2]>(k: [K, K2, K3], v:T[K][K2][K3]): TypedMap<T>;
}

declare var m: TypedMap<{ a: { b: number } }>;

var m2 = m.set(['a', 'b'], 42);

[agalazis]

Yes that's true but it should happen in a recursive way that's what I tried in my example above but since we cannot have spread operator on the array, it's not doable I guess.

[agalazis]

I think a possible solution could be blocked by: https://github.com/Microsoft/TypeScript/pull/17884

[marshall007]

In addition to the libraries mentioned in the initial comment, something like this proposal is absolutely necessary in order to provide any sort of type checking on queries in the elasticsearch.js client driver.

The other very common use case I keep stumbling on is dot notation in query string parameters.

[Nimelrian]

This is an eagerly awaited feature for me! It would allow for much less headaches when dealing with utility functions designed to pull nested properties out of other types!

[Morglod]

Just implemented this feature https://github.com/Morglod/ts-pathof

const c = { z: { y: { bb: 123 }}};
const path = pathOf(c, 'z', 'y', 'bb');
// path now is typeof [ 'z', 'y', 'bb' ]

const path2 = pathOf(c, 'z', 'y', 'gg'); // error, because no 'gg' field in c.z.y

Also types only approach:

let path: pathOf3<typeof c, 'z', 'y', 'bb'>;
path = pathOf(c, 'z', 'y', 'bb');

[KiaraGrouwstra]

Thanks for posting in the other thread, @Morglod. For reference to others, #12290 also discusses tackling similar functions.

fwiw, tackling m.set('a.b', 42) is far off currently -- the only operation we can do using string literal types is pretty much direct object navigation. for type-safety, it may currently be more realistic to stick with tuple-based variants (-> ['a', 'b']) like in e.g. Ramda.

I feel this problem is more general than just path-based navigation though -- similar challenges exist for lenses/traversals as well. Given that, I would advocate a more generic approach based on recursion, which I hope will gain further support.

@agalazis: I think #17884 actually ended up superseded by a recently merged PR by ahejlsberg! :)

[agalazis]

@tycho01 was it released? looking forward to giving it a shot

[Morglod]

@tycho01 just updated ts-pathof.

Now you can:

import { hasPath } from 'ts-pathof';

const c = { z: { y: { bb: 123 }}};
const path = hasPath(c, [ 'z', 'y', 'bb' ]);
path -> [ 'z', 'y', 'bb' ]

const path2 = hasPath(c, [ 'z', 'y', 'gg' ]); // no error
path2 -> value is false, type is never

or

import { PathOf } from 'ts-pathof';

const o = {x: { y: 10 }};

type xy = PathOf<typeof o, ['x', 'y']>;
xy -> ['x', 'y']

type xyz = PathOf<typeof o, ['x', 'y', 'z']>;
xyz -> never

[KiaraGrouwstra]

@agalazis: check out #24897. while technically it doesn't go as far, Concat from https://github.com/Microsoft/TypeScript/pull/24897#issuecomment-401423920 (not officially supported) does get you there as a ... alternative.

[agalazis]

@Morglod still you need 277 lines of code to achieve what we could achieve in just a few (in my example) if spreading keyOf was supported (and to be fair I am not sure how your code copes with arbitrarily nested objects or if it has some sort of limit)

[Morglod]

@agalazis There will be much more code in typescript compiler, server etc to achive this)
I posted it as temporary solution for this case. Better wait 2+ years?

Actually you are right, this hack works only for 2 levels deep.
Rewrited implementation without hacks.

[agalazis]

@Morglod Your work is amazing(I would use your library any time), my point was that the proposal is still valid, I am sure you agree

[Morglod]

@agalazis ye it should be in language

[diegohaz]

This would solve a lot of problems.

[jogibear9988]

we do it like this: (in a class query)

 Filter<Y>(fieldFunc:  (part: T) => Y, filter: FilterDataGridFilterType | "==" | "!=" | "<" | ">" | "in" | "<=" | ">=", value: Y, level?: number) {
    let field = fieldFunc.toString().split('=>')[1].split('.').splice(1).join('.');
    ....

so we could use like this:

    let qry = (new QueryT(BasicActionDTO)).Service(this.mfcname);
    qry = qry.Filter(x => x.Archive, "==", 0);

so we could convert our c# code to typescript nearly the same... we know that it's not safe if someone uses a complex function instead of only property access, but it works for us

[ScreamZ]

Any news or related thread?

That's really problematic in some kind of libraries, such as MongoDB. Where dot notation is used to atomically change fields on a document.

Also on Lodash get/set, ramda and various other libraries.

[thw0rted]

I've been working on a solution to this problem using template string types. Playground link

It's a little convoluted but I was trying to avoid full-on recursion in hopes of suffering from less of a performance penalty. I think it works but may still be slow.

I asked on the TS Gitter and was given a much simpler solution by user @webstrand

type PathOf<T, K extends string, P extends string = ""> =
  K extends `${infer U}.${infer V}`
    ? U extends keyof T ? PathOf<T[U], V, `${P}${U}.`> : `${P}${keyof T & (string | number)}`
    : K extends keyof T ? `${P}${K}` : `${P}${keyof T & (string | number)}`;

declare function consumer<K extends string>(path: PathOf<KnownObjectType, K>);

Note that the second generic parameter is the path string itself, so explicitly passing it is basically redundant -- const myPath: PathOf<SomeObject, "a.b.c"> = "a.b.c". But when you use it to type a function argument, as with consumer above, the path-string type K can be inferred contextually and you can just call consumer("a.b.c") by itself.

I think the pros and cons between my original solution come down to capabilities and performance. My solution fully computes four-deep paths, which I'm sure is slow -- possibly awful if the input type has lots of index signatures etc. Webstrand's solution requires partially-specified generic params to be useful, which is always kind of a hassle, but likely performs a lot better because it's not forcing the compiler to compute the full set of possible key names.

(As an aside, I'd still love to see an officially supported solution for this problem.)

[thw0rted]

In case anybody is tracking this, here's a refined Path type that should be more robust:

type StringableKey<T> =
    T extends readonly unknown[]
        ? number extends T['length']
            ? number : `${number}`
        : string | number;

type Path<T> =
  T extends object
    ? {
        [P in keyof T & StringableKey<T>]: `${P}` | `${P}.${Path<T[P]>}`;
      }[keyof T & StringableKey<T>]
    : never;

Like the other one, I would expect this to perform poorly when used on a very deep or complex input type, and to error out completely if the type is recursive.

[agalazis]

@thw0rted I didn't know about template string types, when was it introduced? Can it be used as a discriminant property in a tagged Union type?

[thw0rted]

It's a relatively new feature. I haven't tried it with tagged unions. I think in most cases it's supposed to work pretty similarly to a literal union, i.e. type Digit = 0|1|2|3|4|5|6|7|8|9; type FooDigit = `Foo${Digit}`; expands to type FooDigit = "Foo0" | "Foo1" | "Foo2" | "Foo3" | "Foo4" | "Foo5" | "Foo6" | "Foo7" | "Foo8" | "Foo9". So, I'd expect it to behave the same way as a literal union when used as a discriminant property.

I played with it a little and it looks like a pretty good result. You can try it for yourself if you like.

[thw0rted]

In case anybody tracking this issue is curious, I asked for a built-in helper type in #46337 and got shot down because helper types can be kind of controversial. As I said over there, maybe a keyword would be better in the long run since the team could really think about how to make it performant, how to handle recursion, how to improve error messages, etc etc. Still watching with interest!