Set up CI system to create desktop installer of vanilla Theia

thegecko commented 5 years ago

Feature requirement to extend CI systems to build installers for the desktop flavour of Theia for:

Windows
MacOS
Linux

In order to correctly sign these, they will need to be built on Windows, Mac and Linux machines accordingly.

Options for systems to include:

Custom Jenkins setups (already proven at arm) available from the Eclipse foundation
Hosted build service for each OS (e.g. AppVeyor for Windows)

cc @arekzaluski @svenefftinge

sr229 commented 5 years ago

@thegecko FWIW, Travis now has a Windows VM for you to use. You may also use Azure pipelines as well.

thegecko commented 5 years ago

Thanks. The Mac ci may be tricky

kittaakos commented 5 years ago

use Azure pipelines

+1

The Mac ci may be tricky

We have not had problems producing a dmg for macOS with Azure so far.

thegecko commented 5 years ago

We have not had problems producing a dmg for macOS with Azure

Have you tried signing the image (if we care about that)?

kittaakos commented 5 years ago

Have you tried signing the image

No.

sr229 commented 5 years ago

At this point we can have our CI migrated to Azure Pipelines since it supports MacOS, Windows and Linux already @thegecko @kittaakos

akosyakov commented 5 years ago

How about using GitHub actions instead? https://github.blog/2019-08-08-github-actions-now-supports-ci-cd/#fast-ci-cd-for-any-os-any-language-and-any-cloud

kittaakos commented 5 years ago

GitHub actions instead

Does it support downloadable artifacts without creating a new release? Azure does.

Otherwise, I do not have any objections.

sr229 commented 5 years ago

GitHub CI is still in beta so we'll need to wait for their november GA.

thegecko commented 5 years ago

How about using GitHub actions

I believe GH actions use docker containers behind the scenes, so are Linux only.

This would mean we couldn't sign installers for Mac / Windows :(

akosyakov commented 5 years ago

It was just a suggestion, I would be happy with anything working :)

thegecko commented 5 years ago

I would be happy with anything working

I get the hint :p

chrisjj commented 4 years ago

Any progress on this?

thegecko commented 4 years ago

No progress as yet, many options have been suggested (Azure, Travis), but as I only have experience building and signing using Jenkins servers more help will be needed if we take a different approach.

As I see it, there are 3 options:

Setup up Jenkins servers taking advantage of the eclipse foundation build systems. This approach builds on a known release strategy we use for Mbed Studio, but requires access to eclipse servers and may diverge our CI from what we already have
Create something from scratch using Travis. I would need some support from someone with Travis foo as I don't have time to research/learn it :). We can use existing Jenkins scripts as a starting point.
Create something from scratch using Azure. Again, I would need some support from someone with Azure foo as I don't have time to research/learn it :). We can use existing Jenkins scripts as a starting point.

Any solution we choose needs to support native MacOS, Windows and Linux build machines in order to sign any installers (if that's a requirement).

Thoughts/volunteers?

akosyakov commented 4 years ago

I somehow like to reuse eclipse build systems. I think it is fine that it is a separate CI, bundling from monorepo does not work anyway, correct? It would be also close to what others have to do. Could scripts be open sourced?

thegecko commented 4 years ago

We can sanitize the scripts and share them to use as a base line. Do we have a contact at eclipse to access their build system?

benoitf commented 4 years ago

opening a bugzilla to get CI instance at https://ci.eclipse.org/theia ?

chrisjj commented 4 years ago

Thanks all.

thegecko commented 4 years ago

https://bugs.eclipse.org/bugs/show_bug.cgi?id=559794

marcdumais-work commented 4 years ago

Any solution we choose needs to support native MacOS, Windows and Linux build machines in order to sign any installers (if that's a requirement).

Just a quick mention that Travis CI meets this requirement of supporting CI for our 3 platforms. Not sure about The Foundation's CI infrastructure - I guess they'll let us know in the bugzilla above.

thegecko commented 4 years ago

Travis CI meets this requirement of supporting CI for our 3 platforms

Excellent, I just need a volunteer with travis foo :)

marcdumais-work commented 4 years ago

Excellent, I just need a volunteer with travis foo :)

A few of us have played with Travis and can help.

When we talk about "vanilla Theia", do we mean the Electron Example application from the main repo, e.g. built and packaged nightly?

We have this example electron app that can already generate native packages for our 3 platforms (manually for now, not included in repo's CI). The electron-builder config could be used as an example, e.g. used along with the Electron Example app's package.json.

arekzaluski commented 4 years ago

Yes. https://github.com/theia-ide/theia-apps/blob/master/theia-electron/electron-builder.yml is a really good start. After 1.0.0 it may be worth adding more improvements. For example:

Enabling Asar (It will require few changes in Theia code. Quite important to reduce the disk size and installation time of Theia)
Updating electron-builder version (There are few known issues in newer versions that may break backward compatibility)
Adding electron-updater for auto-updates

marcdumais-work commented 4 years ago

There will be some preliminary tasks before we get to the CI/Travis aspect:

for each of our supported platforms, chose which packaging(s) we want. Note: For Linux there are packagings that are distro-specific (e.g. .deb, .rpm) and one that's generic (AppImage).
for each wanted packaging, ideally find a public registry where we can push the Theia app, once packaged. e.g. for Docker images, we use dockerhub. If in some cases we can't find such a public registry, we can consider using a generic file hosting service (e.g. Eclipse Foundation infrastructure maybe).
- create an account on each of these registries. Ideally they can provide a revocable authentication token that we can use to authenticate during CI, before pushing. We can add the tokens as Travis environment variables, like we did for dockerhub and npmjs

thegecko commented 4 years ago

We have this example electron app that can already generate native packages for our 3 platforms

I think this is the ideal candidate

There will be some preliminary tasks before we get to the CI/Travis aspect:

IMO, getting the installers building and signing will be the hardest part here. The content and publish location doesn't necessarily block work in this area. I'm happy to share some (potentially redacted!) Jenkins scripts to support anyone trying this on Travis. Let me know where to put them.

for each of our supported platforms, chose which packaging(s) we want. Note: For Linux there are packagings that are distro-specific (e.g. .deb, .rpm) and one that's generic (AppImage).

After a lot of research and trial/error on our side, we recommend the following:

Linux .deb (and .sh) installer containing an AppImage as it's far easier to upgrade
Windows .msi (in an .exe)
MacOS .pkg

for each wanted packaging, ideally find a public registry where we can push the Theia app, once packaged. e.g. for Docker images, we use dockerhub. If in some cases we can't find such a public registry, we can consider using a generic file hosting service (e.g. Eclipse Foundation infrastructure maybe).

DockerHub makes sense for the docker images (shall we create one now?). For the installers, we could simply add them to the GitHub releases on Theia?

create an account on each of these registries. Ideally they can provide a revocable authentication token that we can use to authenticate during CI, before pushing. We can add the tokens as Travis environment variables, like we did for dockerhub and npmjs

Sensible.

akosyakov commented 4 years ago

Most frequent complain for last 2 days after Theia 1.0: https://dev.to/abuassar/comment/n7h0

I'm keeping saying that we target adopters not end users, but having installer won't harm.

hsheikhali commented 4 years ago

@akosyakov I agree with you wholeheartedly. But I believe you can still fish for adopters with installers. Even if it's a simple "hey check out Theia as a desktop app".

thegecko commented 4 years ago

I'm not going to have time to dedicate to this for the next Month or so at least I'm afraid.

chrisjj commented 4 years ago

But I believe you can still fish for adopters with installers.

Installer would be nice, but I'd settle for just a working .exe.

I gave up looking for an official example .exe and the best substitutes I found were Arm mbed and Arduino Theia-based IDE. Given how poor those bests are by the standards of desktop software, an official example IDE (assuming it is better) would avoid discouragement of potential adopters likewise diverted to those substitutes.

hsheikhali commented 4 years ago

@chrisjj Agreed. @thegecko mentioned he didn't have time for this for the next month -- thing about open source is that someone else can pick up that project :)

Crystal-RainSlide commented 4 years ago

Typically, the core of Electron apps is an .asar file, if the compile environment is limited but with a local Electron runtime, one can just produce the .asar file, and run it with the Electron runtime itself.

If there is any problem on the platforms, or I'd like to say, "the Chromium part", and the .asar file can run swiftly on the native Electron (VSCode and Atom's .asar files can't. They modified the Chromium part without writing something try...catch in the JavaScript), then there is a choice to release the .asar file of Theia for those who already got / would like to download an Electron runtime.

This sinks the file size, and also costs less energy. But the result is either not a installer nor a portable program, and there is no visible sign that Electron would support the .asar file and encourage people to download a runtime and make use of the .asar files directly. We still need the installers, and maybe some portable versions / solutions in the future.

ride4sun commented 4 years ago

I would love to try a more flexible layout than VSCODE but it is really not understandable that there is no action on this that an end-user can install this on the mayor three platforms like VSCODE. It seems like there is a big interest and momentum on this.

chrisjj commented 4 years ago

Three months and still there is apparently no published build of vanilla desktop Theia. I'm hoping that's because such a build isn't possible, since that's probably more fixable than the alternative, being that leading contributors it not worth providing.

On Monday, 25 May 2020, ride4sun notifications@github.com wrote:

I would love to try a more flexible layout than VSCODE but it is really not understandable that there is no action on this that an end-user can install this on the mayor three platforms like VSCODE. It seems like there is a big interest and momentum on this.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/eclipse-theia/theia/issues/5481#issuecomment-633336455, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABO7GEPU66H5TMQG7P5L53TRTHD4NANCNFSM4HYJHQQQ .

chrisjj commented 4 years ago

Three months and still there is apparently no published build of vanilla desktop Theia. I'm hoping that's because such a build isn't possible, since that's probably more fixable than the alternative, being that leading contributors it not worth providing.

Correction: that leading contributors consider it not worth providing.

On Monday, 25 May 2020, chrisjj notifications@github.com wrote:

Three months and still there is apparently no published build of vanilla desktop Theia. I'm hoping that's because such a build isn't possible, since that's probably more fixable than the alternative, being that leading contributors it not worth providing.

On Monday, 25 May 2020, ride4sun notifications@github.com wrote:

I would love to try a more flexible layout than VSCODE but it is really not understandable that there is no action on this that an end-user can install this on the mayor three platforms like VSCODE. It seems like there is a big interest and momentum on this.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/eclipse-theia/theia/issues/5481# issuecomment-633336455, or unsubscribe https://github.com/notifications/unsubscribe-auth/ ABO7GEPU66H5TMQG7P5L53TRTHD4NANCNFSM4HYJHQQQ .

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/eclipse-theia/theia/issues/5481#issuecomment-633557504, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABO7GEKTAVOIX3ID2O764S3RTJSZFANCNFSM4HYJHQQQ .

marcdumais-work commented 4 years ago

@chrisjj This is open-source - PRs are welcome

chrisjj commented 4 years ago

You'd think so. Yet still this one hasn't appeared.

On Monday, 25 May 2020, Marc Dumais notifications@github.com wrote:

@chrisjj https://github.com/chrisjj This is open-source - PRs are welcome

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/eclipse-theia/theia/issues/5481#issuecomment-633591811, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABO7GEM7BR72CEWQXFVMI7DRTJ4G3ANCNFSM4HYJHQQQ .

sr229 commented 4 years ago

@chrisjj Instead of complaining, why don't you take initiative and make the PR to support the builds? Desktop builds are possible but its something the team isn't yet willing to provide since there are more important issues to tackle for now (such as better extension support).

ride4sun commented 4 years ago

I am wondering if that is not done (better extension support) yet why even that big announcement for version 1.0. Maybe that was a bit premature. I thought Theia is a competition of VSCODE. The strength of VSCODE is multiplatform. The weakness of VSCODE is the non-flexible Layout and lousy multiple screen support. Big announcement but sorry - you can not try it out on your code. (Really)

Sorry, an installer is something I would just call "basic feature". There is always something more important.

vince-fugnitto commented 4 years ago

I am wondering if that is not done (better extension support) yet why even that big announcement for version 1.0. Maybe that was a bit premature. I thought Thea is a competition of VSCODE. The strength of VSCODE is multiplatform. The weakness of VSCODE is the non-flexible Layout and lousy multiple screen support. Big announcement but sorry - you can not try it out on your code. (Really)

Sorry, an installer is something I would just call "basic feature". There is always something more important.

@ride4sun based on your comment it looks like you might have misunderstood what Theia is. Theia is in fact a framework or platform to build IDE-like applications and it is not an IDE in itself (like vscode). The project allows application-developers to build and customize their own IDE-like products while also providing the flexibility and extensibility which allow for a wide range of use-cases.

Having an installer for the 'default' application (ex: example-browser or example-electron applications present in the repo) are actually nice-to-haves and is not a 'basic feature's like you mentioned.

ride4sun commented 4 years ago

Sorry, but it sounded a good alternative to VSCODE to me when I was reading the announcements:

https://www.globenewswire.com/news-release/2020/03/31/2009010/0/en/The-Eclipse-Foundation-Releases-Eclipse-Theia-1-0-a-True-Open-Source-Alternative-to-Visual-Studio-Code.html

Quote: Theia is designed from the ground to run on both Desktop and Cloud... Eclipse Theia is designed to work as a native desktop application as well as in the ...

Native Desktop which oh sorry you can not install because it is not important enough?

Even the title says that: The Eclipse Foundation Releases Eclipse Theia 1.0, a True Open Source Alternative to Visual Studio Code

So let me get the correct... it is an alternative I can just not install it????

Quote: (https://devclass.com/2020/03/31/eclipses-theia-sees-1-0-declared-not-your-parents-ide/) Mike Milinkovich, executive director of the Eclipse Foundation, said in a statement: “Visual Studio Code is one of the world’s most popular development environments. Not only does Theia allow developers to install and reuse VS Code extensions, it provides an extensible and adaptable platform that can be tailored to specific use cases.”

So I can install an extension like in VSCODE but I can not install THEIA???

chrisjj commented 4 years ago

@chrisjj https://github.com/chrisjj Instead of complaining, why don't you take initiative and make the PR to support the builds?

I've no idea how to make such a PR. Plus as I said, the fact that far more able persons than I haven't done it suggests it is much more difficult that you imply.

C

chrisjj commented 4 years ago

ride4sun, let's add:

The most significant differences between Eclipse Theia and VS Code are:

Theia's architecture is more modular and allows for significantly more customizations
Theia is designed from the ground to run on both Desktop and Cloud
Theia is developed under the community-driven and vendor-neutral governance of the Eclipse Foundation.

which at the moment should also include

Theia doesn't give you an .exe because that's not important enough.

C

vince-fugnitto commented 4 years ago

@ride4sun if you look at the quotes you referenced you'll see the following:

Not only does Theia allow developers to install and reuse VS Code extensions, it provides an extensible and adaptable platform that can be tailored to specific use cases.

In the EclipseFoundation announcement you'll see:

The Eclipse Foundation, one of the world’s largest open source foundations, today announced the release of Theia 1.0, a true open source alternative to Microsoft’s popular Visual Studio Code (VS Code) software. Eclipse Theia is an extensible platform to develop multi-language Cloud and Desktop Integrated Development Environments (IDEs) with state-of-the-art web technologies that enable developers, organizations, and vendors to create new, extensible developer experiences.

It clearly illustrates that the project is a framework which can build IDE-like applications which in fact rival and is an open-source alternative to VS Code. The main target audience for the framework are application-developers who are in need of a framework to build their own white label IDE-like applications.

The idea of providing an executable or installer to end-users is for the ability for more of an audience to get a feel of the default functionality and behavior of the framework but once again it will only be one example of many possible use-cases. Nothing prevents you today to build you own application using Theia.

In any case, the addition of desktop installers for an example Theia application, is being actively worked on by the main community and hopefully it will bring forth more visibility and users to the framework.

ride4sun commented 4 years ago

I worked on many software projects myself and I have to admit that the installer is something nobody wants to work on. I have the feeling that is the non-outspoken reason here.

That is really stupid because I really want to try the new layout engine and you guys should be eager to get a big install base to gain momentum and get feedback.

ride4sun commented 4 years ago

@vince-fugnitto

Let make it clear than....

Theia 1.0, a true open source alternative to Microsoft’s popular Visual Studio Code

This is NOT a Vscode alternative because it addresses another use case. Totally misleading!

vince-fugnitto commented 4 years ago

@ride4sun

I worked on many software projects myself and I have to admit that the installer is something nobody wants to work on. I have the feeling that is the non-outspoken reason here.

That is really stupid because I really want to try the new layout engine and you guys should be eager to get a big install base to gain momentum and get feedback.

@ride4sun it is being worked on :) Providing end-users with installers is something the community wants to ensure is done right, and the necessary planning, background tasks, setting up the CI jobs and permissions.

Let make it clear than.... This is NOT a Vscode alternative because it addresses another use case. Totally misleading!

It can be argued that it is an open-source alternative since vscode is not truly open-source, we are providing vscode extension support, we have worked on providing an open-source alternative to the marketplace for downstream applications to use (open-vsx), and the default commands, implementation and user-interface are vscode inspired.

If you disagree then that is totally your opinion, I think the multiple comments (re-iterating the same point) are sufficient for this particular issue.

chrisjj commented 4 years ago

It clearly illustrates that the project is a framework which can build IDE-like applications ...

It would do... if it were not contradicted by e.g. "a True Open Source Alternative to Visual Studio Code".

The fact Theia is a platform for building IDEs whereas VSCode is an IDE is probably the most important entry that should be on Theia's "The most significant differences between Eclipse Theia and VS Code are..."

None of which bears on the deficiency of an actual .exe for vanilla Theia.

It can be argued that it is an open-source alternative since vscode is not truly open-source

No. That's merely makes Theia a more open-source non-alternative.

C

ride4sun commented 4 years ago

I agree with that statement but that is quite a different statement than the press releases.

It can be argued that it is an open-source alternative since vscode is not truly open-source, 
we are providing vscode extension support, we have worked on providing an open-source
alternative to the marketplace for downstream applications to use (open-vsx), and the default commands, implementation and user-interface are vscode inspired.

Can you share the community efforts where people working on the installer?

vince-fugnitto commented 4 years ago

Can you share the community efforts where people working on the installer?

The main community has discussed this feature and other features in the weekly dev-meeting.

chrisjj commented 4 years ago

The main community has discussed this feature and other features in the weekly dev-meeting https://github.com/eclipse-theia/theia/wiki/Dev-Meetings.

Any sign of "actively working on"? As per your "the addition of desktop installers for an example Theia application, is being actively worked on by the main community".

On Wed, 27 May 2020 at 00:31, Vincent Fugnitto notifications@github.com wrote:

Can you share the community efforts where people working on the installer?

The main community has discussed this feature and other features in the weekly dev-meeting https://github.com/eclipse-theia/theia/wiki/Dev-Meetings.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/eclipse-theia/theia/issues/5481#issuecomment-634334124, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABO7GELLEYRKQZMNJVMS7MLRTRGMRANCNFSM4HYJHQQQ .

eclipse-theia / theia

Set up CI system to create desktop installer of vanilla Theia #5481