Separate types from code and make types composable/extendable

[gr2m]

Currently all JS Octokit modules depend on @ocotkit/types. @octokit/types include definitions for all 600+ REST API endpoints for api.github.com and primitives such as types for Requests and responses that are not extendable.

Problems

1. All endpoint types are for api.github.com. They cannot be replaced for GHAE or a specific GHES version. They cannot be imported selectively.
2. The types cannot be extended by plugins. Plugins often time introduce new Octokit options such as throttle, extend request options and/or responses. Authentication strategies are a special kind of plugins that provide custom auth options, but they cannot define the options in a way that would provide IntelliSense to users of the authentication strategy. Plugins also hook into the request life cycle and amend requests/responses, but the types for requests/responses provided by @octokit/types cannot be extended at this point.

Solution

Take advantage of declaration merging and global augmentation to make it possible for imported code to amend the global Octokit Types.

For the endpoint types, I see two approaches:

1. Create entirely separate types for api.github.com, GHES-3.0, GHES-3.1
2. Use types for api.github.com as the starting point, then create types for the GHES/GHAE versions as an extension of that. Endpoints that do not (yet) exist in the target version can be set to never with an explanation. If the endpoint exists in both environments but differs in its request parameters or response, create a union of both types. We could introduce a new option such as request.version which defaults to "api.github.com" but which can be set to whatever version of the REST API endpoint types were loaded. So when it's set to "ghes-3.1" then the types from that version are used explicitly, instead of the union between the types of api.github.com and ghes-3.1.

I'd prefer the 2nd approach as I'd expect it to be a better developer experience.

For developers who write code that is meant to run against both api.github.com and a minimal version of GHES, we could create virtual versions such as ghes-3.0-compatible, which will either only include types that are covered by both environments, or make clear which endpoints/properties do not exist in either.

Benefits

• Make developing for GHAE and GHES a first-class experience

Steps

@octokit-next/* packages

• [x] Create a new octokit-next.js repository with the minimal set of features needed to implement the extendable & composable types. I already started working on parts of it in https://github.com/gr2m/javascript-plugin-architecture-with-typescript-definitions/. octokit-next.js could build upon that to test the composability features of Base.defaults() and Base.plugins() and implement a .request() method which implements a few of GitHub's endpoints to test a setup that works against different environments (api.github.com, ghes-3.1, ghes-3.0)
  • [x] extendable constructor options: https://github.com/gr2m/javascript-plugin-architecture-with-typescript-definitions/pull/56
  • [x] inherit constructor options types when calling .defaults(): https://github.com/gr2m/javascript-plugin-architecture-with-typescript-definitions/pull/57
  • [x] Create octokit-next.js repository with an octokit.request method which utilizes endpoint types
  • [x] https://github.com/octokit/octokit-next.js/issues/3
• [x] Implement types for 3-5 endpoints that have differences across api.github.com, ghes-3.1, ghes-3.0. Make the types for api.github.com load by default.
• [x] Add options.version parameter for the Octokit constructor and a options.request.version option when calling octokit.request(options). The type should be an extendable enum. By default it will only permit api.github.com. When importing types for ghes-3.0, it can be set to api.github.com and ghes-3.0 (and if it makes the implementation easier because of the cascade: ghes-3.1)
• [x] https://github.com/octokit/octokit-next.js/issues/18
• [x] https://github.com/octokit/octokit-next.js/issues/14
• [x] https://github.com/octokit/octokit-next.js/issues/15
• [x] https://github.com/octokit/octokit-next.js/issues/16
• [x] https://github.com/octokit/octokit-next.js/issues/17
• [x] https://github.com/octokit/octokit-next.js/issues/36
• [x] https://github.com/octokit/octokit-next.js/issues/41
• [x] octokit/octokit-next.js#70
• [x] octokit/octokit-next.js#72
• [x] octokit/octokit-next.js#63
• [x] octokit/octokit-next.js#88
• [ ] ESM-ify all official authentication strategies
• [ ] ESM-ify octokit
  • [ ] @octokit/oauth-methods
  • [ ] @octokit/oauth-app
  • [ ] @octokit/webhooks-methods
  • [ ] @octokit/webhooks
  • [ ] @octokit/app
  • [ ] @octokit/plugin-paginate-rest
  • [ ] @octokit/plugin-rest-endpoint-methods
  • [ ] @octokit/plugin-retry
  • [ ] @octokit/plugin-throttling

Implement changes in Octokit

• [ ] Setup octokit/types-rest-api.ts repository as a monorepo that publishes all the @octokit/types-rest-api-* packages with all the update-automation
• [ ] Migrate @octokit/types to the new version in the beta branch
• [ ] Migrate @octokit/endpoint (the lowest-level @octokit/* package that uses the endpoints types) in the beta branch
• [ ] Migrate @octokit/request in the beta branch
• [ ] Migrate @octokit/graphql in the beta branch
• [ ] Migrate @octokit/core in the beta branch
• [ ] Migrate the plugins loaded by octokit. Extend Octokit.Options as needed
• [ ] Migrate the @octokit/* authentication strategies
• [ ] Migrate octokit
• [ ] Migrate or rather re-create @octokit/plugin-enterprise-server to utilize the new types. Only generate methods for the latest GHES version that do not exist on github.com

Backlog

• [ ] remove the default GET / route types. Instead implement an override of request(route, parameters) that only applies if Octokit.ApiVersions["github.com"] is an empty object (if that is possible at all). Use that override's description to recommend installing one of the types packages

[gr2m]

This will unlock https://github.com/octokit/core.js/issues/323

[gr2m]

For the curious ones, here's what this is all about

https://user-images.githubusercontent.com/39992/129640719-48be7f8c-2fce-4e74-9556-3794a73e291b.mp4

[gr2m]

Another update, in 4 parts, because

1. There is more to show
2. I'm giving TypeScript a hard time

Part I: upcoming @octokit/core without any REST API types and with types for github.com

https://user-images.githubusercontent.com/39992/133866362-2b758476-ee5f-4c4b-b81c-d3f545126154.mp4

Part II: types for GitHub Enterprise Server (GHES)

https://user-images.githubusercontent.com/39992/133866363-e6ce9da9-f9c8-436a-ab4e-7c394eb14130.mov

Part III: set API version on new Octokit() constructor

https://user-images.githubusercontent.com/39992/133866368-f7256212-32ee-413a-82b7-b9c4779636b9.mp4

Part IV: types for GHES-compatible versions

https://user-images.githubusercontent.com/39992/133866371-a4e636ef-a6d9-4684-9159-a8eb1b9fe1ab.mp4