A Script to Scaffold Applications around Extensions

[colin-grant-work]

The basic goal here is to make the process of developing, consuming, and forking extensions developed outside the main Theia repo smooth, preferably with a workflow very similar to working on Theia itself. The suggestion below is one idea, but certainly not the only approach.

At the moment, the process of developing and consuming Theia packages from outside this repository is not very smooth. Two that I regularly work with, the Theia Trace Extension and Theia CPP Extensions both come wrapped in their own example applications. That can allow for some customization, but comes with the overhead of ensuring that the wrapper is up to date with Theia as well as the risk of getting far enough away from Theia that the developer experience working on the extension is quite different from working on a package inside Theia - making for a steeper learning curve and unnecessary friction.

One solution would be a Theia extension script that could scaffold a full Theia application around an extension for demos / testing and then unwrap the extension again.

Feature Description:

Something like this:

1. Before wrapping:

   └── myExtensionRepo
   ├── .gitignore
   ├── package.json
   └── src
       ├── browser
       ├── common
       └── node

2. execute yarn wrap = 'theia-ext-scripts wrap'
3. See this:

   └── myExtensionRepo
   ├── .gitignore
   ├── package.json
   ├── src
   │   ├── browser
   │   ├── common
   │   └── node
   └── wrapper
       ├── examples
       │   ├── browser # With some @theia packages specified by myExtensionRepo/package.json
       │   └── electron
       └── packages
           └── my-extension
               └── ... # Content here copied or symlinked from myExtensionRepo/...

   Where the wrapper is .gitignored, and everything necessary for the package to operate in the example applications is copied - or, better, symlinked - from its location in the upper folder.

This would have a number of benefits:

1. Working on a standalone Theia extension would look a lot more like working on a VSCode extension - just the source code of the extension.
2. Developers who wanted to start a Theia extension could take an existing package from Theia as their model, and just take queues for any changes in config from Theia, reducing integration friction.
3. Ensuring that their extension worked with latest Theia would only mean keeping their ext-scripts dependency up to date and generating the wrapper.
4. The repo would be coterminous with the extension, so adopters who wanted to fork the code could just include that repo as a submodule in their own packages folder and set about working as normal. At the moment, the fork has to live somewhere outside the repo and be linked in with some work in the tsconfig.
5. Encourage relative uniformity of convention across Theia applications. There would have to be some thought given to how to let teams customize their configurations, but the path of least resistance would be to follow Theia's conventions, which come with the wrapper, and that would make it easier to work on different extensions, upstream good parts of extensions or integrate them whole into the Theia repo. This is already partly accomplished by existing ext-script functionality, but it could be carried further.

[colin-grant-work]

@paul-marechal, this is the write up of an idea that we talked about the other day. Not sure it's clearly expressed, but I want to flag the issue of difficulties developing and consuming standalone extensions for consideration - exactly where we end up is pretty open.

[colin-grant-work]

Something like git's sparse checkout functionality could help with forking standalones, but doesn't encourage uniformity across extensions.

[tsmaeder]

@colin-grant-work in my experience, relying on symlinked sources is not reliable: the problem is this: tools some tools manipulate paths as paths and others as strings: what I mean is this: when you have a path into a symlinked directory /bar/zoz (with zoz being symlinked from /foo/zoz), some tools will treat /bar/zoz/../bla as /bar/bla while others will us /foo/bla. Tool writers don't seem to expect sources to be symlinked 🤷