Suggestion for cooperation

[Jarthianur]

Hello together,

first to say I really like the theia project and this repository is excellent to get theia to run. Inspired by your work, I've developed a little tool which attempts to provide a more generic and structured way of configuring and building theia applications. You can find it at https://github.com/Jarthianur/theia-workspace-builder.

I don't really want to advertise it, but you may find it interesting though. As it requires quite a lot of effort to maintain all those applications and languages - especially if you use only a small part of it by your self -, we could maybe come together in some sort of cooperation. For example, you could use the tool for providing the docker applications and in turn help with developing the required "modules".

[vince-fugnitto]

cc @marcdumais-work @marechal-p

[vince-fugnitto]

@Jarthianur thank you for your interest in the project and your initiative in developing an alternative solution to the standard docker setup we currently have as part of the repo. We will take some time to understand the proposal, and determine if it suits our needs.

A little bit of backstory, this repository is used for continuous-integration and as standalone examples for the community to possibly inspire themselves to build their own images (I would not say the development effort is substantial at the moment). If we are to use the builder, we somewhat lose the standalone examples and the community will need to ramp up on the builder (which we will also need to maintain and add new modules). Since we likely do not want to remove the standalone docker images present in the repo, we will need to discuss if we also want to commit to additional effort to support the builder as well. It is something we will need to consider as a community, and take the time to weigh the pros and cons.

[Jarthianur]

Thanks for your fast answer. I completely understand your point.

[...] this repository is used for continuous-integration and as standalone examples for the community to possibly inspire themselves to build their own images [...] - I would say mission accomplished.

Just to add a bit of information, there would be no need to discard the standalone examples attitude. The builders main task is to prepare Dockerfiles and package.json files for the application. These could be populated in theia-apps just like before. It could also be a catch for #274 .

[vince-fugnitto]

he builders main task is to prepare Dockerfiles and package.json files for the application.

@Jarthianur do you mind showing me an example for cpp if you have one? I'd like to see what the dockerfile will look like, and how the environment is set up.

[Jarthianur]

Of course...

I have this example at hand. It is just a showcase but provides actually all necessary information. The resulting files are:

package.json

{
  "dependencies": {
    "@theia/callhierarchy": "next",
    "@theia/console": "next",
    "@theia/core": "next",
    "@theia/cpp-debug": "next",
    "@theia/debug": "next",
    "@theia/editor": "next",
    "@theia/editor-preview": "next",
    "@theia/file-search": "next",
    "@theia/filesystem": "next",
    "@theia/getting-started": "next",
    "@theia/git": "next",
    "@theia/keymaps": "next",
    "@theia/markers": "next",
    "@theia/messages": "next",
    "@theia/metrics": "next",
    "@theia/mini-browser": "next",
    "@theia/monaco": "next",
    "@theia/navigator": "next",
    "@theia/outline-view": "next",
    "@theia/output": "next",
    "@theia/plantuml": "next",
    "@theia/plugin": "next",
    "@theia/plugin-ext": "next",
    "@theia/plugin-ext-vscode": "next",
    "@theia/preferences": "next",
    "@theia/preview": "next",
    "@theia/process": "next",
    "@theia/scm": "next",
    "@theia/search-in-workspace": "next",
    "@theia/task": "next",
    "@theia/terminal": "next",
    "@theia/typehierarchy": "next",
    "@theia/userstorage": "next",
    "@theia/variable-resolver": "next",
    "@theia/vsx-registry": "next",
    "@theia/workspace": "next"
  },
  "devDependencies": {
    "@theia/cli": "next"
  },
  "license": "Apache-2.0",
  "name": "@theia/example-theia-application",
  "private": true,
  "theia": {
    "frontend": {
      "config": {
        "applicationName": "Example Theia Application",
        "preferences": {
          "files.enableTrash": false
        }
      }
    }
  },
  "theiaPlugins": {
    "custom-theme": "https://open-vsx.org/api/.../example.vsix",
    "doxygen-generator": "https://open-vsx.org/api/cschlosser/doxdocgen/1.1.0/file/cschlosser.doxdocgen-1.1.0.vsix",
    "marky-markdown": "https://open-vsx.org/api/robole/marky-markdown/2.7.0/file/robole.marky-markdown-2.7.0.vsix",
    "material-icon-theme": "https://open-vsx.org/api/PKief/material-icon-theme/4.5.0/file/PKief.material-icon-theme-4.5.0.vsix",
    "plantuml": "https://open-vsx.org/api/jebbs/plantuml/2.14.0/file/jebbs.plantuml-2.14.0.vsix",
    "prettier": "https://open-vsx.org/api/esbenp/prettier-vscode/5.9.1/file/esbenp.prettier-vscode-5.9.1.vsix",
    "rust-lang": "https://open-vsx.org/api/rust-lang/rust/0.7.8/file/rust-lang.rust-0.7.8.vsix",
    "vscode-builtin-bat": "https://open-vsx.org/api/vscode/bat/1.51.1/file/vscode.bat-1.51.1.vsix",
    "vscode-builtin-configuration-editing": "https://open-vsx.org/api/vscode/configuration-editing/1.51.1/file/vscode.configuration-editing-1.51.1.vsix",
    "vscode-builtin-cpp": "https://open-vsx.org/api/vscode/cpp/1.51.1/file/vscode.cpp-1.51.1.vsix",
    "vscode-builtin-debug-auto-launch": "https://open-vsx.org/api/vscode/debug-auto-launch/1.51.1/file/vscode.debug-auto-launch-1.51.1.vsix",
    "vscode-builtin-docker": "https://open-vsx.org/api/vscode/docker/1.51.1/file/vscode.docker-1.51.1.vsix",
    "vscode-builtin-icon-theme-seti": "https://open-vsx.org/api/vscode/vscode-theme-seti/1.51.1/file/vscode.vscode-theme-seti-1.51.1.vsix",
    "vscode-builtin-image-preview": "https://open-vsx.org/api/vscode/image-preview/1.51.1/file/vscode.image-preview-1.51.1.vsix",
    "vscode-builtin-ini": "https://open-vsx.org/api/vscode/ini/1.51.1/file/vscode.ini-1.51.1.vsix",
    "vscode-builtin-json": "https://open-vsx.org/api/vscode/json/1.51.1/file/vscode.json-1.51.1.vsix",
    "vscode-builtin-json-language-features": "https://open-vsx.org/api/vscode/json-language-features/1.51.1/file/vscode.json-language-features-1.51.1.vsix",
    "vscode-builtin-log": "https://open-vsx.org/api/vscode/log/1.51.1/file/vscode.log-1.51.1.vsix",
    "vscode-builtin-make": "https://open-vsx.org/api/vscode/make/1.51.1/file/vscode.make-1.51.1.vsix",
    "vscode-builtin-markdown": "https://open-vsx.org/api/vscode/markdown/1.51.1/file/vscode.markdown-1.51.1.vsix",
    "vscode-builtin-markdown-language-features": "https://open-vsx.org/api/vscode/markdown-language-features/1.51.1/file/vscode.markdown-language-features-1.51.1.vsix",
    "vscode-builtin-merge-conflict": "https://open-vsx.org/api/vscode/merge-conflict/1.51.1/file/vscode.merge-conflict-1.51.1.vsix",
    "vscode-builtin-perl": "https://open-vsx.org/api/vscode/perl/1.51.1/file/vscode.perl-1.51.1.vsix",
    "vscode-builtin-powershell": "https://open-vsx.org/api/vscode/powershell/1.51.1/file/vscode.powershell-1.51.1.vsix",
    "vscode-builtin-rust": "https://open-vsx.org/api/vscode/rust/1.51.1/file/vscode.rust-1.51.1.vsix",
    "vscode-builtin-search-result": "https://open-vsx.org/api/vscode/search-result/1.51.1/file/vscode.search-result-1.51.1.vsix",
    "vscode-builtin-shellscript": "https://open-vsx.org/api/vscode/shellscript/1.51.1/file/vscode.shellscript-1.51.1.vsix",
    "vscode-builtin-sql": "https://open-vsx.org/api/vscode/sql/1.51.1/file/vscode.sql-1.51.1.vsix",
    "vscode-builtin-theme-defaults": "https://open-vsx.org/api/vscode/theme-defaults/1.51.1/file/vscode.theme-defaults-1.51.1.vsix",
    "vscode-builtin-xml": "https://open-vsx.org/api/vscode/xml/1.51.1/file/vscode.xml-1.51.1.vsix",
    "vscode-builtin-yaml": "https://open-vsx.org/api/vscode/yaml/1.51.1/file/vscode.yaml-1.51.1.vsix",
    "vscode-clangd": "https://open-vsx.org/api/llvm-vs-code-extensions/vscode-clangd/0.1.7/file/llvm-vs-code-extensions.vscode-clangd-0.1.7.vsix",
    "vscode-editorconfig": "https://open-vsx.org/api/EditorConfig/EditorConfig/0.16.4/file/EditorConfig.EditorConfig-0.16.4.vsix",
    "vscode-references-view": "https://open-vsx.org/api/ms-vscode/references-view/0.0.77/file/ms-vscode.references-view-0.0.77.vsix"
  },
  "theiaPluginsDir": "plugins",
  "version": "0.0.1"
}

Dockerfile

FROM archlinux/archlinux:latest

ARG THEIA_UID=1337

RUN pacman -Syyuu --noconfirm && \
    pacman -S --needed --noconfirm \
      yarn nodejs-lts-erbium npm tar grep awk wget git openssh make gcc python2 vim bash-completion binutils && \
    pacman -Scc --noconfirm && \
    npm config set python /usr/bin/python2 --global && \
    npm config set python /usr/bin/python2

RUN useradd -s /bin/bash -m -d /home/theia -U -u $THEIA_UID -c '' theia && \
    chmod g+rw /home && \
    mkdir -p /home/project && \
    chown -R theia:theia /home/theia && \
    chown -R theia:theia /home/project

RUN pacman -S --needed --noconfirm sudo && \
    pacman -Scc --noconfirm && \
    echo 'theia ALL = NOPASSWD: /usr/bin/pacman' >> /etc/sudoers

RUN pacman -S --needed --noconfirm \
      rustup && \
    pacman -Scc --noconfirm

RUN su - theia -c "rustup default stable" && \
    su - theia -c "rustup component add rust-src rust-docs rls rust-analysis rustfmt clippy"

RUN pacman -S --needed --noconfirm \
      clang \
      gcc \
      cmake \
      doxygen \
      gdb \
      valgrind \
      cppcheck && \
    pacman -Scc --noconfirm

RUN wget -q https://github.com/linux-test-project/lcov/releases/download/v1.15/lcov-1.15.tar.gz && \
    tar xf lcov-1.15.tar.gz && \
    cp -n lcov-1.15/bin/* /usr/local/bin/ && \
    rm -rf lcov-1.15*

ADD settings.json /home/theia/.theia/
ADD .gitconfig /home/theia/

RUN pacman -S --needed --noconfirm \
      python-pip && \
    pacman -Scc --noconfirm
RUN su - theia -c "pip install --user compiledb"
RUN echo "hello world"

RUN chown -R theia:theia /home/theia

USER theia
WORKDIR /home/theia
ADD package.json ./package.json

ENV PATH=/home/theia/bin:/home/theia/.local/bin:$PATH

RUN yarn --pure-lockfile && \
    NODE_OPTIONS="--max_old_space_size=4096" yarn theia build && \
    yarn theia download:plugins && \
    yarn --production && \
    yarn autoclean --init && \
    echo *.ts >> .yarnclean && \
    echo *.ts.map >> .yarnclean && \
    echo *.spec.* >> .yarnclean && \
    yarn autoclean --force && \
    yarn cache clean && \
    rm -f .yarnclean

ENV THEIA_DEFAULT_PLUGINS=local-dir:/home/theia/plugins \
    SHELL=/bin/bash \
    VISUAL=vim \
    EDITOR=vim

EXPOSE 3000

ENTRYPOINT [ "node", "/home/theia/src-gen/backend/main.js", "/home/project", "--hostname=0.0.0.0" ]

Based on the base image used, it may look different, as the archlinux build is intended to be a cutting edge one.

[marcdumais-work]

Hi @Jarthianur ,

I had a quick look at your repo and, at first look, I think it's amazing work!

Given the direction of the project, now supporting programming languages through vscode extensions only, I think we might be able to simplify by removing the Theia extension contributed for each language in favor of adding the few remaining in the base application template. If the community decides to also generally supply debug features through vscode extensions, some existing ones might disappear. E.g. @theia/cpp-debug might then be deprecated in favor of enhancing and using cdt-gdb-vscode. To simplify further, we might consider building and packaging a "standard" Theia browser app, and re-use it in various images along with specific vscode extensions, without having to rebuild a new application each time.

Can I ask about the Apache 2.0 license: did you select it because you copied some content from this repo? Other than Theia-related bits you may have copied from this repo, is the content all developed by you? Would you be amenable to license your work under the Eclipse Theia project license?

This repo here pre-dates our project joining the Eclipse Foundation. We have a new repo under there, called theia-blueprint, where we have a more formal example application that we call a product template. For now it's desktop-only but we want/are thinking to add a new image, from scratch but of the same general idea as the Full image there. I think a consideration would be to propose your workspace builder as a tool that extends blueprint.

[Jarthianur]

Hi @marcdumais-work,

thanks for your praise. I mean actually it's just a bit of templating around json and dockerfiles.

I like the idea of packaging a single standard theia base application. For now, I took the approach of completely splitting things up into their respective modules to be consistently modular, but putting the basic stuff together wouldn't hurt.

About the license, well I copied some of the parts related to docker and theia packaging. The builder tool itself, as well as most module installs are my very own work. Hence there is nothing preventing me from switching to EPL v2 license.

[marcdumais-work]

The builder tool itself, as well as most module installs are my very own work. Hence there is nothing preventing me from switching to EPL v2 license.

Awesome, thanks for the reply @Jarthianur

As it requires quite a lot of effort to maintain all those applications and languages - especially if you use only a small part of it by your self -, we could maybe come together in some sort of cooperation. For example, you could use the tool for providing the docker applications and in turn help with developing the required "modules".

Understood - it's a lot of work. Theia apps tend to be hard to keep in build-able shape.

We (@vince-fugnitto and I) can only contribute to repos/projects under theia-ide (legacy) and eclipse-theia and I do not think your generator ATM solves a problem that's high on our employer's list. I am unsure how much the community of current committers would be interested. Probably no one would object on principle, but I do not know either that anyone would commit to help maintaining it.

If you feel comfortable in doing so, you could join us at one of our dev-meetings and present your work. This would draw attention to your generator and possibly interest for it. If you let me know in advance I can add an entry to the agenda.

Else i'll mention what we're working on a way to package vscode-builtins using vscode extension packs. This will make the vanilla case a lot easier to manage, e.g. pulling all builtins from one package.json entry instead of ~70. We are also working on "constant-URL" support for vscode extensions, that would install the latest version likely to work. With this the built-in URLs don't need to be updated to new versions.

These two things should simplify your modules eventually. In the meantime do not hesitate to open issues here if you have questions or problems.

[Jarthianur]

Thanks for your reply and inviting me to the dev-meetings.

Well to be honest, I don't think the builder would add any value to theia itself. I can absolutely understand, that supporting such a tool is not of any priority to theia devs and related companies.

In my opinion theia-apps, and blueprint also, are quite important to the "average user", as it gives you something at hand to actually work with. You know, one can customize and build a language specific application easily, just by looking at how it is done here. And this is where the builder could add some value. A unified and structured way of providing these "apps", and even add more apps to the place.

But I'm fine with it so far. No need to promote stuff at dev-meetings. When you have any further ideas or plans with the builder, especially if I should switch the license, feel free to reach out.

So long, thanks for this awesome project!

[stale[bot]]

This contribution has been automatically marked as stale due to inactivity, and it will be closed if no further activity occurs. Thank you for contributing to Theia!

[marcdumais-work]

Thanks @Jarthianur . Closing this issue,