Write a function to define types.

ruojianll commented 3 years ago

Search Terms

type defination, functional type defination, meta, meta programming, dynamic type declaration

Suggestion

Write a function to define types.

Use Cases

Build a more complex type.

Examples

type MyType<T extends string>{
  if(T extends 'a'){
    return 'a' //type 'a'
  }
  if(T extends 'b'){
    return 'b' //type 'b'
  }
  return string
}

Checklist

My suggestion meets these guidelines:

[x ] This wouldn't be a breaking change in existing TypeScript/JavaScript code
[x ] This wouldn't change the runtime behavior of existing JavaScript code
[x ] This could be implemented without emitting different JS based on the types of the expressions
[x ] This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, etc.)
[x ] This feature would agree with the rest of TypeScript's Design Goals.

shrinktofit commented 3 years ago

Interesting. It's more like C++'s meta programming. Another approach is allow constant value to be passed as generic parameter:

type Conditional<Condition: boolean, T, U> = Condition ? T : U;

type MyType<T extends string> = Conditional<T extends 'a', 'a', Conditional<T extends 'b', 'b', string>>;

MartinJohns commented 3 years ago

Related #39385, where someone suggested evaluating user-code as part of the compilation, to which the TypeScript team responded:

Executing user code during compilation time is the first step toward madness; we don't implement features like this and don't intend to.

jcalz commented 3 years ago

Does this proposal differ in some meaningful way from just using conditional types?

type MyType<T extends string> = T extends 'a' ? 'a' : T extends 'b' ? 'b' : string;
type A = MyType<'a'> // 'a'
type B = MyType<'b'> // 'b'
type C = MyType<'c'> // string

Is it just asking for a different syntax for existing functionality? Or is there something else going on here?

weswigham commented 3 years ago

If the key part of the OP was more-JS-like composition of type manipulations, I had a proof of concept of something kinda like this in this branch, enabling, for example:

type function StringConcat(a: ts.StringLiteralType, b: ts.StringLiteralType) {
    return this.createLiteralType(a.value + b.value);
}

type Res = StringConcat<"hello", "world">;

complete with language service support within the type functions. It's notable in that it's more like inline compiler-plugin-esque type providers than strictly statement-level new type syntax, like the OP seems to suggest. There's a bunch of unanswered questions on how it should work in many areas, even if you get over determining if the feature is worth the complexity in the first place.

adamvictorclever commented 3 years ago

I'm not completely sure if this is what the OP is referring to, but I'd be interested in some alternate syntax like the one in the OP with similar/identical functionality to what can be expressed using conditional types. Conditional types can get very unwieldy when there are lots of nested ternaries and it would be nice to be able to more clearly write them using a function-like syntax.

AlCalzone commented 3 years ago

@adamvictorclever so https://github.com/microsoft/TypeScript/issues/41123 then?

ruojianll commented 3 years ago

Related #39385, where someone suggested evaluating user-code as part of the compilation, to which the TypeScript team responded:

Executing user code during compilation time is the first step toward madness; we don't implement features like this and don't intend to.

It's not the same thing as running user code in compiling time in exsiting program languages. I suggested a safe and full constraint syntax to build types, it's not javascript or running in javascript runtime.

ruojianll commented 3 years ago

Maybe we need a general imperative type difination language to enable us to help compiler build types in compiling time not only use declarative syntax.

mistlog commented 3 years ago

I implemented typetype so that we can generate typescript type easily. It's still primitive, however, it really works.

For example, in the url-parser example, the input would be such as:

type function parseURL = (text) => ^{
    if (parseProtocol<text> extends [infer protocol, infer rest]) {
        return {
            protocol,
            rest
        }
    } else {
        return never
    }
}

or

type function _isNumberString = (text) => ^{
    if(text extends "") {
        return true
    } else if(text extends `${infer digit}${infer rest}`) {
        return ^{
            if(digit extends Digit) {
                return _isNumberString<rest>
            } else {
                return false
            }
        }
    } else {
        return false
    }
}

ruojianll commented 3 years ago

If the key part of the OP was more-JS-like composition of type manipulations, I had a proof of concept of something kinda like this in this branch, enabling, for example:
type function StringConcat(a: ts.StringLiteralType, b: ts.StringLiteralType) {
    return this.createLiteralType(a.value + b.value);
}

type Res = StringConcat<"hello", "world">;
complete with language service support within the type functions. It's notable in that it's more like inline compiler-plugin-esque type providers than strictly statement-level new type syntax, like the OP seems to suggest. There's a bunch of unanswered questions on how it should work in many areas, even if you get over determining if the feature is worth the complexity in the first place.

@weswigham It's seems useful in my scene. Is there more information?

owl-from-hogvarts commented 3 years ago

Does this proposal differ in some meaningful way from just using conditional types?
type MyType<T extends string> = T extends 'a' ? 'a' : T extends 'b' ? 'b' : string;
type A = MyType<'a'> // 'a'
type B = MyType<'b'> // 'b'
type C = MyType<'c'> // string
Is it just asking for a different syntax for existing functionality? Or is there something else going on here?

With such functions, i guess, we will get ability to constract types from parsed static structures such as template strings. Consider the example:

// we want know what template params we have for that string
const templateString: string = "/some/parametrised/path/:id"

meta function metaParseTemplateString(str: string) {
  // implementation code which will parse string and figure out what params it has
  // then it will construct type represeting these parameters
  // and return them like:
  return type {
    id: string
  }
}

// somewhere later
const paramsForTemplateString: metaParseTemplateString(templateString) = {
  id: "someIdA154ds"
}

// but
const test: metaParseTemplateString(templateString) = {
  // error
  foo: 'asd'
}

owl-from-hogvarts commented 3 years ago

@ruojianll, i advise you to add next Search Terms:

meta, meta programming, dynamic type declaration

ruojianll commented 2 years ago

@ruojianll, i advise you to add next Search Terms:
meta, meta programming, dynamic type declaration 

@owl-from-hogvarts Updated. Thank for your advice.

mindplay-dk commented 1 year ago

Someone recently posted a "mental model of TS types" on Twitter - this tries to relate the declarative type expression syntax to imperative code, which is what JS developers know.

I think this feature ought to be considered quite seriously. Only a few people ascend to become the rare sort of TypeScript Gurus who can wrangle the type system, which is not only a departure from the syntax of the language it's building on top of, but also an entirely different language paradigm: essentially a touring-complete functional programming language on top of JS.

It's more like C++'s meta programming.

It is, but it's also not.

C++'s meta programming generates code - whereas what's proposed here generates types only.

Related #39385, where someone suggested evaluating user-code as part of the compilation, to which the TypeScript team responded:

Executing user code during compilation time is the first step toward madness; we don't implement features like this and don't intend to.

But they already did.

Someone implemented Worldle using TS types.

Here's an article explaining how to build tic-tac-toe using TS types.

So it's already a fully functional programming language for types.

TS type expressions are functions that operate on types.

Is it just asking for a different syntax for existing functionality?

From my perspective, it is.

We're asking for a syntax that's more familiar and accessible to JS developers.

I don't think we're asking for TS to become a meta-programming language, where you can write JS that actually generates JS - what I have in mind is JS that generates TS types, using a more familiar syntax, and using a paradigm that's more accessible to JS developers.

I think part of the reason TS divides the community somewhat, is because typing is hard. I have 24 years of experience as a web developer. I have no substantial experience with functional programming languages, I've had to work hard on this for years, and I'm still nowhere near grasping more advanced examples like the Wordle and tic-tac-toe examples above.

I still frequently find myself running into problems I just can't solve - I've had to shelve many interesting ideas that were easy to prototype in JS, but despite my best efforts, I could not figure out how to make them type safe. I know many developers in the same boat, and most of them do not go to the lengths I'm willing to go to.

Libraries such as ts-toolbelt has tried to make this more accessible, by implementing many of the core functional primitives as types - and definitely succeeds to some degree. But if you don't have at least a bit of experience with FP languages, you're still likely going to feel somewhat lost and overwhelmed by the learning curve here.

That's why I think this feature is a great idea - possibly the greatest idea ever proposed for TS.

Can you imagine bridging that gap. Shutting down the last legit excuse to write code without types. "Typing is hard". Yeah, it is. It's really, really hard. Maybe it's time to consider a feature that would make it really easy and accessible to Everyone ? 🙂

I think you have a real chance to bridge divide here.

brillout commented 1 year ago

This is especially true for library authors.

As a TypeScript end-user, you can always bail out with any.

But as a library author, you want to provide type safety which more often than not involves very complex types. I've already spent a lot of time trying to provide type safety to my users.

This would be a dramatic and foundational change. Digging into this could be tremendously worth it.

owl-from-hogvarts commented 1 year ago

Let's make this not just a discussion but a solid proposal.

Goal

Construct complex types with easy

Examples

Build type to allow any non-abstract subclass of abstract base class.

As of now this is quite difficult task:

// Allows only concrete constructable versions of the abstract class,
// but preserves any properties (e.g. statics) directly attached to T
export type ConstructableSubclass<
  T extends abstract new (...args: any[]) => unknown,
  Constructor extends new (...args: any[]) => InstanceType<T> = new () => InstanceType<T>
> = Constructor & Omit<T, never>;

With functions this vastness would be replaced with nice, easy to understand type function with familiar syntax

Validate template string literals

By now this is not possible at all

But with type functions:

// we want know what template params we have for that string
const templateString = "/some/parametrised/path/:id" as const

type function metaParseTemplateString<S extends string> {
  // implementation code which will parse string and figure out what params it has
  // then it will construct type representing these parameters
  // and return them like:
  return new Type({
    id: "string"
  })
}

// somewhere later
const paramsForTemplateString: metaParseTemplateString<templateString> = {
  id: "someIdA154ds"
}

// but
const test: metaParseTemplateString<templateString> = {
  // error
  foo: 'asd'
}

Concepts and definitions

Syntax

Reasonable syntax would be:

type function name<param1, param2 extends SomeOtherType> {}

Type parameters are values of type Type (more on that type later). Therefore all js syntax is valid within type functions.
Parameters has support of all current generic features like extends statements etc. to impose restrictions right in function signature.
Function should return Type

What is `Type`?

Type is a complete description of some ts type. As type functions will operate on values of Type they should be present as object (exact set of properties can be determined later):

type function ConstructableSubclass<T> {
  if (T.typeof === "function") {
    if (T.parameters[0].typeof === "number") {
      const config = new Type({
        size: "number",
        foo: {
          foobar: new Field({
            type: new Type("function", {
              params: {
                param1: "number",
                param2: "string"
              }, 
              returnType: "string",
            }),
            optional: true
          })
        }
      })

      T.parameters.unshift(config)
      return T
    } else {
      return T
    }
  } 
  return "never"
}

To be able to use existing type features inside functions, types created with Type should be available as actual types and not just values. And actual types should be reachable in values context:

type function MakeOptional<T extends {}> {
  const foobar = {...T, asdasd: new Field({type: "string"})}
  // here Readonly is available in value context
  const joitom = Readonly<foobar>
  // here foobar is available in type context
  type optional = Partial<foobar>
  return optional
}

Thus typescript would need to prepare the code for execution that is compile types to objects and emit valid js.

Possible problems

Functions need to be executed somehow. Typescript by itself cant execute type function code. Thats why code should be executed by runtime, in which TS is executed. But that would mean that untrusted code will be run in full environment. Therefore we need to build wallen garden. And this is not that easy, because typescript should stay runtime agnostic i.e. should work with both deno and node and probably web.

While for node and deno it is certainly possible with native modules such as isolated-vm. For deno we may just deny all permissions. But I doubt that this will be secure enough. Alike solutions are runtime dependent, so ts will lose ability to be run almost everywhere.

Thats why we should consider webworkers. They are isolated from main thread by default, already providing better security. But they still have some api available. Remember that we want to allow only pure javascript. Therefore we need to block all apis except Array, Math and so on. However this still requires some runtime dependent code, because node has it's own worker api (see web-worker npm).

mindplay-dk commented 1 year ago

@owl-from-hogvarts thanks for getting the ball rolling! This is a great start. 🙂

Functions need to be executed somehow. [...] this is not that easy, because typescript should stay runtime agnostic i.e. should work with both deno and node and probably web.

What I would suggest here, is type functions should only have access to the standard ECMAScript globals and fundamental types/objects.

The language standard library doesn't expose anything security sensitive, as far as I'm aware - while Node and browser APIs would provide access to storage and network, the standard language run-time only provides things like string, number, date, functions, objects, symbols, etc. Providing access to these shouldn't be an issue - and should provide more than enough functionality to compute new and interesting types that couldn't (at least not easily) be computed before.

Furthermore, type functions should be able to call type functions only - they should not be able to call regular functions defined in your code. Strict separation of compile-time and run-time artifacts should make this feature easier to implement as well. (I don't think isomorphism between the compile-time and run-time would even be useful here, as compile-time functions are going to accept and return types - which won't exist or make sense outside of the compile-time environment.)

I think it's important to keep in mind that type functions would be types - these functions would not exist outside of the compile-time, and if you wanted to create a library of type functions, these would need to ship as a .d.ts file (via "types" in package.json) and would only be available to the compiler. Similar to how type libraries work currently.

I do not think we should permit importing of referencing regular functions (from npm packages, or from anywhere) and calling those from type functions. At least not in a first version of this feature.

With regards to the Type API and structured types, I would suggesting avoiding this kind of API:

new Type({ foo: ... })

Representing an object type { foo: string } using a similarly structured object type such as { foo: new Type("string") } seems convenient on the surface, but I would suggest representing those as structures with well-defined properties instead.

Rather object types with constructors, I would also suggest representing all types using a single Type interface with a discriminator.

So for example, let's say you want to construct the following type:

type MyType = {
  foo: string,
  bar: {
    baz: string[]
  }
}

I would propose we construct this entirely out of value types:

type function MyType {
  return {
    type: "object",
    properties: [
      {
        name: "foo",
        type: "string"
      },
      {
        name: "bar",
        type: {
          type: "object",
          properties: [
            {
              name: "baz",
              type: {
                type: "array",
                element: "string"
              }
            }
          ]
        }
      }
    ]
  }
}

An internal schema for types might look something like this:

type Type = "string"
  | "number"
  | "boolean"
  // ...
  | ObjectType
  | ArrayType;

type ObjectType = {
  type: "object",
  properties: Array<{
    name: string;
    // ...
    type: Type;
  }>,
}

type ArrayType = {
  type: "array",
  element: Type,
}

// ...

Note that value-typed structures are JSON-serializable, which could become interesting in the future.

mindplay-dk commented 1 year ago

You can see this kind of structure being type-checked in a playground here.

mindplay-dk commented 1 year ago

Thinking about this some more... 🤔

Functions need to be executed somehow. Typescript by itself cant execute type function code. Thats why code should be executed by runtime, in which TS is executed. But that would mean that untrusted code will be run in full environment. Therefore we need to build wallen garden.

It just occurred to me, all of this is true of TypeScript itself, now - the situation is not that different. The main difference is, they're building a custom runtime for the type system. This is "untrusted code" as well.

Of course, it doesn't have access to a full environment - as you say, we would need to build a "walled garden", but that seems mostly to be a matter of deciding which APIs will be available. I don't believe anything in the standard JS run-time provides any sort of access to anything unsafe - e.g. nothing in the language standard JS APIs exposes any network or IO related APIs, afaik.

And this is not that easy, because typescript should stay runtime agnostic i.e. should work with both deno and node and probably web.

Deno and browsers already have the standard language-defined APIs available, so that's likely not a problem? Another issue to consider would be non-standard compilers, such as ESBuild or Babel - but third party compilers generally don't do type-checking at all, so this would likely mostly consist of parsing and ignoring any new syntax, same as any new TS feature.

What made me think about this again today was this presentation:

https://speakerdeck.com/zoontek/advanced-typescript-how-we-made-our-router-typesafe

It's quite an interesting case study - it walks through a lot of TS typing features, and if you've heard some people saying "TS feels like a different programming language", this is likely the sort of thing they're talking about.

You're basically reimplementing the string based parser functions of this router in a pure functional recursive language-within-the-language - the reasoning, programming style and syntax being entirely foreign to JS itself.

I think this makes a very strong case for exploring this feature.

It also makes a good case study for the proposal: can we port this router to proposed type-functions? This would prove it's complete, as well as demonstrating the difference in terms of ergonomics and usability in real-world production code. (Fancy routers like this are everywhere in TS frameworks and libraries by now.)

This particular examples gets me thinking about the type function literals proposed by @owl-from-hogvarts above. Do we need dedicated syntax for type functions? What if TS functions were simply callable in type expressions? Regular function declarations have the required syntax already, I think. Maybe some sort of syntax to denote function calls could be added to type expressions, e.g. some kind of escape character before the function name?

Perhaps the type function syntax could be optional, and would only mean "don't emit this function, and don't allow it to be called as a regular function" - so there would be a way to denote functions to be used exclusively in types.

But wouldn't it be practical, if e.g. the parser functions required by a router could be implemented once, as regular TS functions - and you could merely call them from type expressions somehow?

Some of the arguments for this feature include familiarity, ease of use, nearness to the JS language itself, and so on - but another great argument would be actually eliminating duplicate code. As of right now, some parts of a router are implemented twice, in (essentially) two completely different languages and styles - imagine simply writing your router, then calling the string functions from types to apply the exact same transformations statically without duplicating anything!

ruojianll commented 10 months ago

I still need this now.

mindplay-dk commented 7 months ago

I wonder if type functions would be somehow applicable to solving this little problem?

https://twitter.com/jamonholmgren/status/1751409105644982694

I scrolled this thread for half an hour before throwing in the towel - one solution is worse than the next, an absolute circus of complexity for such a trivial requirement.

There has got to be a better way. 🤔

jedwards1211 commented 4 months ago

More use cases:

Computing input/output types for GraphQL queries
String parsing ala ArkType
Numeric comparison types that work for numbers of any magnitude, and don't require goofy mapping like type Successor = { 0: 1, 1: 2, 2: 3 }

ruojianll commented 1 month ago

So would we do something about this? I still need something to build complex types! A deeply nested extends syntax is verbose and difficult to read.

microsoft / TypeScript