[Proposal] Migration from JavaScript to TypeScript

[Desplandis]

This issue is a feature proposal. Feel free to upvote (with :+1: ), comment and provide your use-cases if you're interested by this feature.

Context

The iTowns codebase is currently written in Javascript, with JSDoc comments providing a loose form of type-checking that isn't enforced (but in practice used by the LSP of our code editors). This lack of type safety has led to side-effects creeping into the project, making maintenance and debugging more challenging and causing unpredictable behavior in certain areas of the code.

This proposal outlines a migration path to TypeScript, which will enforce type safety, simplify the refactoring of complex or difficult-to-maintain parts of the code, and enable developers to take full advantage of modern tooling and best practices.

Description of the proposal

This proposal details the steps for migrating iTowns from JavaScript to TypeScript. The migration will follow a systematic, bottom-to-top approach, beginning with simpler, independent modules before tackling more complex and interdependent ones.

Along the way, we will take the opportunity to refactor key components to improve clarity and maintainability. These refactorings will be addressed through separate proposals and pull requests.

This issue will primarily serve as a tracker for the migration process and a space to discuss each phase of the migration (see tracking section below).

Implementation

Tooling

The migration will necessitate some changes on our tooling:

• Modify the ESLint configuration to support TypeScript, integrating the @typescript-eslint plugin. We should ensure that both JavaScript and TypeScript code follow a unified set of rules, with TypeScript as a superset of JavaScript.
• Modifiy both babel and webpack config to support TypeScript as input (this should be relatively easy).
• Transition from JSDoc to Typedoc for documentation, as JSDoc does not support TypeScript, and Typedoc will provide stricter type-driven documentation.

Code migration

The migration process will be gradual, but the following steps could serve as a good start:

• Migrate geographic modules: migrate geographic modules first, since they have been identified as simple and easy to convert, and could provide a good first case to ensure that our tooling is correct.
• Refactor and split the Extent module: split Extent into two sub-modules—one for geographic extents and one for tiled extents. This refactor is long due, as these modules currently contribute to at least 25% of all type issues and need a cleaner structure for proper typing.
• Refactor Tile Geometry Creation: refactor and migrate the tile geometry creation mechanism, which has accumulated several layers of patches and requires simplification to improve maintainability.
• Loaders and Parsers: migrate the various loaders to TypeScript. We could either let them as if, or follow the Three.js convention for loaders (i.e. converting them to classes inheriting Three.Loader) and use a global Three.LoadingManager to share loaders between layers (and maybe facilitate Draco loader setup). Those are not however the most fragile parts of iTowns (except the vector parsers which depends on a lot of dependencies).

Along the way

We could on our migration path rework certain parts of iTowns, those are some thoughts for now and they have not yet been formalized.

• Unit-test each migrated modules
• Remove 2-years+ old deprecated code: this should be straightforward.
• Remove all providers of iTowns (related to #1001, possibility to have a global fetch pool, rework the scheduler)
• [...]

Potential Problems

• Documentation: Moving from JSDoc to Typedoc will likely require rewriting parts of our current documentation, as TypeDoc enforces stricter rules. Moreover, this shift could lead to significant changes in the layout and structure of the generated HTML documentation.
• Backward compatibility: There may be challenges ensuring backward compatibility during the migration process. Since we currently lack a clear distinction between private and public APIs, we will need to evaluate each case individually to determine whether certain changes should be considered breaking (and/or just go for a 3.0 version of iTowns).

Documentation

• Current dependency tree: the green nodes are modules without dependencies, blue ones with dependencies and red ones with circular dependencies.

Tracking list

The tracking list will be written from our discussions, as a list of pair (Migration step, reference to issue and/or PR).

Final note

Feel free to extend and refine each of these points.

I will open a test PR on this subject to further explore the logistics of the migration and identify any overlooked aspects or missing steps.

[mgermerie]

On the documentation matter, do JSDoc and TypeDoc share a documentation generated format or will we need need a temporary template (for the duration of the migration) that supports both generated documentations ?

[Desplandis]

It does not support old JSDoc templates (unfortunately or not?). There seems to have a well-defined API to hooks some plugins as well as themes which seems to be collections of plugins + hooks on the generated HTML and styling.

[jailln]

:+1: :+1: In addition to migrating to typescript, it will force us to clean the codebase, improve the doc and separate public and private API. A long but necessary process.

I'm wondering what's the better way to do it:

• we all work on it for a short period of time and tackle it at once
• we do it gradually (e.g. with objectives each month for instance): is this feasible?

Also, we should consider api-extractor instead of ts-doc, see this remark - no strong opinion for me yet.