search

eclipse-theia / theia

Eclipse Theia is a cloud & desktop IDE framework implemented in TypeScript.
http://theia-ide.org
Eclipse Public License 2.0
19.87k stars 2.48k forks source link

Browser-Only: Run Theia without a backend #12852

Open sdirix opened 1 year ago

sdirix commented 1 year ago

image

Current state

File management

Features

Testing

Documentation

tl;dr;

We would like to enable a new use case for Theia applications: Support for "browser-only" Theia applications which can be executed without a backend.

Our vision is to support Theia "browser-only" applications as a first class citizen, next to the "browser" and "electron" targets. For this we adapted the Theia CLI tooling to support the new browser-only target and frontendOnly Theia extensions. Using this tooling the feature can be supported in a minimally invasive way, requiring none (or little to none) changes to existing Theia based applications.

Check it out yourself

Because it's so easy to host static sites, we deployed three example applications for you to check out:

Details

Click here for the original issue text including high level overview of the feature # Motivation Support of browser-only use cases is a value adding feature for the whole Theia ecosystem. While the existing Theia framework offers large flexibility and therefore supports a wide array of use cases, it requires the deployment and execution of a Node based backend. This often comes with a lot of operational effort and additional costs. There are many use cases in which not the full feature set of Theia based applications are required. For example, demo and quick start applications often come with hard coded examples anyway, do not require persistent storage and can work without terminal support. This feature is requested from time to time in the community too: - https://github.com/eclipse-theia/theia/issues/8368 - https://github.com/eclipse-theia/theia/issues/6732 - https://github.com/eclipse-theia/theia/issues/6560 - https://community.theia-ide.org/t/does-theia-work-without-any-backend/46 - https://community.theia-ide.org/t/would-it-be-possible-to-create-a-backend-less-theia/1545 - https://community.theia-ide.org/t/can-i-build-theia-as-a-browser-front-end-widget-only/646 - https://community.theia-ide.org/t/client-only-ide-for-typescript/1622 # Implementation Theia's tooling already generates the frontend separately from the backend in the `lib/frontend` directory. So technically the frontend can already be served with any web server. However, the frontend will try to establish a web socket connection to its backend and many services will not work when their proxy counterparts are not available. ## Tooling changes We added a new target for Theia applications: `browser-only`, e.g ```json "theia": { "target": "browser-only" } ``` We added a new entry for Theia extension modules: `frontendOnly`, e.g. ```json "theiaExtensions": [ { "frontend": "lib/browser/i18n/i18n-frontend-module", "frontendOnly": "lib/browser-only/i18n/i18n-frontend-only-module", "backend": "lib/node/i18n/i18n-backend-module" }, ] ``` For this target, the Theia CLI (`theia build`) will: - skip generating the backend - for `src-gen` the `frontend`/`frontendOnly` pairs are handled in the same way as `backend`/`backendElectron` are, i.e. if both are present, `frontend` will be skipped. Using these new tools we were able to replace, mock and/or remove all services which need to connect to the Theia backend. `theia start` is supported for browser-only targets too: It will launch the frontend with the `webpack-dev-server`. ## Implementation changes Using the tooling described above we were able to support the new browser-only target almost without any changes to the existing code base. The current [PR only contains two breaking changes](https://github.com/eclipse-theia/theia/pull/12853/files#diff-06572a96a58dc510037d5efa622f9bec8519bc1beab13c9f251e97e657a9d4ed): A rename of a class, and the move of some classes from the `node` directory to the `common` directory. All other changes are only contained within `frontendOnly` modules. The following subsections describe the most important features: ### Filesystem Since there is no native file system available in the browser, we settled on using [BrowserFS](https://github.com/jvilk/BrowserFS), as it offers a Node compatible API, is feature rich and is well supported. We foresee that every browser-only Theia application might want to handle the filesystem in a different way, for example to exclusively bundle some hard coded files, handling persistence via webworker cache, local storage, IndexedDB or only in memory. Therefore, we added a new service, the `BrowserFSInitialization`, that can be customized to support any behavior. In the PR, we used this service in the new "browser-only" example application to initialize the file system with some basic example JSON files. ### WebSocketConnectionProvider The `WebSocketConnectionProvider` of Theia is used to create proxy services for Theia's RPC mechanism. Via our proposed PR, we already support a minimal Theia without using any proxy services consisting of `@theia/core`, `@theia/filesystem`, `@theia/editor`, `@theia/getting-started`, `@theia/keymaps`, `@theia/markers`, `@theia/monaco`, `@theia/navigator`, `@theia/outline-view`, `@theia/preferences`, `@theia/property-view`, `@theia/userstorage`, `@theia/variable-resolver` and `@theia/workspace`. To enable using all Theia packages, even if they still depend on `WebSocketConnectionProvider`, we rebind `WebSocketConnectionProvider` to a mock implementation whose connections will always timeout. This allows using all Theia packages without further changes, even though the functionality will be broken wherever it relies on the backend, e.g. the file search will never complete. Eventually we would like to see all Theia packages supporting their full feature set (as far as possible) for the browser-only case as well. Once that is the case, we technically no longer need a mocked `WebSocketConnectionProvider` in the browser-only frontend, however it might make sense to still keep it for 3rd-party dependencies which are not adapted yet. #### Outlook For performance reasons it could make sense to move some services to web workers to lessen the load on the main Javascript thread of the frontend. For this use case an alternative `ConnectionProvider` could be implemented which bridges the gap to the web worker instead of to the backend. # Open Tasks We think it already makes sense to merge this feature into the Theia code base via this [PR](https://github.com/eclipse-theia/theia/pull/12853). It's already possible to create specific feature-complete browser-only applications for certain use cases. The situation will only become better with more features being iteratively enabled over time. In communication with the community it must be made clear that a "browser-only" Theia can't support all use cases of a regular "browser" or "electron" Theia and that for now this feature shall be considered experimental. We collected a non-exhaustive list of tasks which can be tackled as a follow up of the current PR. **File management** - Finish the BrowserFS filesystem provider - Implement file watching - Implement directory deletion - Implement file renaming - Missing browser-only feature: Import Files into workspace (context menu > Upload files…) - Missing browser-only feature: Download files from workspace (context menu > Download) - Missing browser-only feature: Search in files **Features** - Support Theia plugins and VS Code web extensions - for more information see the following section - Support user preferences - Workspace preferences are already supported via BrowserFS filesystem - Adapt remaining pieces of preloader - Support terminal / console - Support localization - Support source control - Support debugging - Support OpenVSX registry - Support tasks - Or remove feature if not applicable - Show list of extensions in about dialog - Support log levels - Support `theia test` for browser-only targets **Documentation** - Add documentation for browser/browser-only modules in https://theia-ide.org/docs/architecture#separation-by-platform ## Theia plugins and VS Code web extensions support The largest missing piece is Theia plugins and VS Code web extensions support. The good news is that we already have a POC showcasing that the support of Theia plugins and VS Code web extensions is conceptually feasible: - You can find the hosted POC [here](https://plugins--browseronly-theia.netlify.app/) - You can execute a `hello world` command implemented by a Theia plugin - By opening a Typescript file you can see the Typescript VS Code web extension at work - You can clone Github repositories via the `File > Import repository` action. Note that this uses the open proxy [cors.isomorphic-git.org](https://cors.isomorphic-git.org/) so it's pretty slow. This proxy is freely available so please be considerate with its usage. - **Note that this POC is not optimized, so it takes a while to start.** - You can find the code on a branch [here](https://github.com/eclipsesource/theia/tree/browser-only-plugins-poc) Conceptually, Theia is well prepared to support Theia plugins and VS Code web extensions in the browser-only case, as these are already executed in the [browser in a webworker](https://github.com/eclipse-theia/theia/tree/f00450f38bfee7d0b7f2c60b46e7f24202bdbb32/packages/plugin-ext/src/hosted/browser). However, it will require some substantial effort until plugin support will be ready for production in the browser-only case: - All plugins are organizationally exclusively handled on the backend side, especially metadata collecting and deploying - Any call to the Theia API / VS Code API which is currently routed through the backend will not work in the browser-only case Even though there is no obstacle for eventually fully supporting Theia plugins and VS Code web extensions in the browser-only case, it will require some work. # Summary With the support of browser-only applications we hope to open an exciting new chapter for the Theia framework. Please have a look at our PR and the hosted example applications and share your opinion below. Thank you!
goyalyashpal commented 8 months ago

so, this "backend-less/browser only" theia is to theia-desktop what github.dev or vscode.dev are to github.codespaces right?

sdirix commented 8 months ago

Yes that's a fair comparison when looking at the "github.dev web based editor". The clarification is needed as GitHub codespaces is also served via the github.dev domain.

goyalyashpal commented 8 months ago

this blog about vscode-dev lists many more (practical) scenarios where the serverless application becomes indispensible

my ~summary~ distillate of it: any scenario where full fledged desktop application can't be installed but a browser is available. so like:

msujew commented 8 months ago

@goyalyashpal For these use cases, it's probably still best to have a full Theia instance running (for example in the cloud). The backend-less Theia version has other use-cases. For example, we're currently working on https://github.com/eclipse-theia/theia/issues/2842 which in conjunction with the backend-less Theia would allow users to join any workspace in the web without needing to setup a whole Theia server instance.

erlmachinedev commented 7 months ago

Extremely valuable contribution 🚀 I do Theia based project from the beginning and must admit that security considerations around backend-less option give me a way better confidence.

You have an option to control everything you want via dedicated service (the great way for virtualization of resources).

P.S. Taking into account extensible design of the tool I would wish (as a user) to see browser-only version first (where the backend comes as an option)

jjkavalam commented 1 week ago

@sdirix Hi, found this while looking for a viable front end option for a desktop app I am planning to build.

https://minimal--browseronly-theia.netlify.app/# is almost exactly what I want as a starting point.

I will be really grateful if you could point me to the repository where the sources for this app can be found.

(I am not looking to use electron for the desktop app itself; i just need the front end that theia provides)

msujew commented 1 week ago

@jjkavalam It's the browser-only example app found in this repo.

jjkavalam commented 1 week ago

@msujew Sorry for not being clear enough.

I want to extract that example out of the repo into an independent directory (basically integrate it into another project that I have).

So I am doing:

mv eclipse-theia/examples/browser-only myapp
cd myapp
yarn

This gives me:

% yarn
yarn install v1.22.22
info No lockfile found.
[1/4] 🔍  Resolving packages...
error Error: https://registry.yarnpkg.com/@theia%2fapi-samples: Not found
    at params.callback [as _callback] (/Users/jobin.kavalam/.nvm/versions/node/v20.17.0/lib/node_modules/yarn/lib/cli.js:66680:18)

Also, I understand that the browser-only project is relying on generated sources.

How to generate them ? After generation can I just move the browser-only directory out as mentioned above and use it independently.

msujew commented 1 week ago

@jjkavalam Please take a look at our documentation on how to build your own app. In addition to the steps outlined there, you need to set the theia.target" setting tobrowser-only` like its done in the example linked above.

jjkavalam commented 1 week ago

@msujew Thank you. I was able to make progress. Following the guide, i have a package.json like below:

{
  "private": true,
  "dependencies": {
    "@theia/callhierarchy": "latest",
    "@theia/file-search": "latest",
    "@theia/git": "latest",
    "@theia/markers": "latest",
    "@theia/messages": "latest",
    "@theia/mini-browser": "latest",
    "@theia/navigator": "latest",
    "@theia/outline-view": "latest",
    "@theia/plugin-ext-vscode": "latest",
    "@theia/preferences": "latest",
    "@theia/preview": "latest",
    "@theia/search-in-workspace": "latest",
    "@theia/terminal": "latest"
  },
  "devDependencies": {
    "@theia/cli": "latest"
  }
}

However, am interested in a very minimal setup similar to https://minimal--browseronly-theia.netlify.app/.

I wonder which packages i could safely remove without breaking anything else ?

msujew commented 1 week ago

@jjkavalam You can likely get away just with @theia/navigator and @theia/monaco.