Closed kdvolder closed 4 years ago
@kdvolder you can use the resolution for latest
images, next
images and apps have been fixed:
I'm sorry but isn't 'latest' supposed to be the stable version whereas 'next' is the bleeding edge?
For the time being we switched to hard pinning us to version 1.0.0 (copying theia artfacts from the 1.0.0 image instead of running theia-build). This seems to be the only one that works somewhat consistently at the moment and doesn't break frequently and unexpectedly as theia versions / dependencies evolve between builds.
I'm sorry but isn't 'latest' supposed to be the stable version whereas 'next' is the bleeding edge?
The latest
build uses the latest
published extensions on npm, while next
uses the bleeding edge. In this case the error was due to a dependency, and occurred after the latest
release. If you are creating a product then the best course of action would be to include a yarn.lock
(like the main repo, vscode, and other products) so you are protected by dependencies updating.
That sounds like a very good advice, thank you. I will look into it.
Perhaps it is thus also a good idea for the theia-java Docker image build here: https://github.com/theia-ide/theia-apps/tree/master/theia-java-docker To do the same? (And similarly for the other images as well)
I.e perhaps for the 'latest' image build a 'yarn.lock' file with "verified as working properly" dependencies locked in place, should be included in this git repo alongside the package.json file.
What do you think?
Or perhaps I am just misunderstanding the intended purpose of 'latest' as referring to a stable and working version? If so, is there a better way for me to get something that does represent a 'stable' build of theia?
That sounds like a very good advice, thank you. I will look into it.
Perhaps it is thus also a good idea for the theia-java Docker image build here: https://github.com/theia-ide/theia-apps/tree/master/theia-java-docker To do the same? (And similarly for the other images as well)
I.e perhaps for the 'latest' image build a 'yarn.lock' file with "verified as working properly" dependencies locked in place, should be included in this git repo alongside the package.json file.
What do you think?
I'm not sure what others will think, I generally don't have an issue with including a yarn.lock
, it will require more effort however to maintain and ensure we update it and test it thoroughly each release (or we can simply pick up the one present in the main repo and apply it to our images). Not having a yarn.lock
does have an advantage however of us identifying issues that may be present which we can fix (such as this particular issue).
Or perhaps I am just misunderstanding the intended purpose of 'latest' as referring to a stable and working version? If so, is there a better way for me to get something that does represent a 'stable' build of theia?
I'd say including a yarn.lock
(similarly to the one present in the main upstream repo) would be the best bet. It is used everyday for development and is updated when needed.
ensure we update it and test it thoroughly each release
I think that would be the point of calling something a 'release'? As consumer of the theia apps, I have some expectation the release is not something that is broken... or breaks after the fact because of dependencies not being locked in and drifting after release.
I'd say including a yarn.lock (similarly to the one present in the main upstream repo) would be the best bet. It is used everyday for development and is updated when needed.
I guess my problem is... where to get a 'good' yarn.lock file that represents the latest 'stable' version.
It is used everyday for development and is updated when needed.
That doesn't sound 'stable' to me.
Not having a yarn.lock does have an advantage however of us identifying issues that may be present which we can fix (such as this particular issue).
I guess that tells me that probably 'latest' isn't really meant to be a 'stable' release (and if it isn't meant to be because of the way you use it... that is totally fine of course). But it does bring me back to my question of where do I as a 'consumer' of theia get a suitable/stable yarn.lock file?
If 'next' isn't considered as a 'stable' release and 'latest' isn't either, then perhaps their should be third set of images / apps that are tagged as 'stable' in addition to these.
ensure we update it and test it thoroughly each release
I think that would be the point of calling something a 'release'? As consumers, I have some expectation the release is not something is broken... or breaks after the fact because of dependencies not being locked in and drifting after release.
I'd say including a yarn.lock (similarly to the one present in the main upstream repo) would be the best bet. It is used everyday for development and is updated when needed.
I guess my problem is... where to get a 'good' yarn.lock file that represents the latest 'stable' version.
The docker images are not official releases, they are used for testing, continuous-integration and examples for others to consult, not official products.
Not having a yarn.lock does have an advantage however of us identifying issues that may be present which we can fix (such as this particular issue).
I guess that tells me that probably 'latest' isn't really meant to be a 'stable' release (and if it isn't meant to be because of the way you use it... that is totally fine of course). But it does bring me back to my question of where do I as a 'consumer' of theia get a suitable/stable yarn.lock file?
If 'next' isn't considered as a 'stable' release and 'latest' isn't either, then perhaps their should be third set of images / apps that are tagged as 'stable' in addition to these.
Again, these aren't official products but examples.
The docker images are not official releases, they are used for testing, continuous-integration and examples for others to consult, not official products.
Fair enough, so it all comes down to me having not understood the 'purpose' of those images.
But my question still remains then... what then, does represent an official release?
Theia 1.1.0 is released. So shouldn't there be an artefact somewhere that 'records' a working yarn-lock file of that releases working dependencies?
My question is, if there is, where do I find it? And if there isn't... then isn't it a bit of a gaping hole in the 'release process' ?
But my question still remains then... what then, does represent an official release?
In the upstream theia repository, we have a release each month based on the latest
changes of all npm extensions. Every release contains multiple changes and we are constantly working towards improving the framework, fixing issues, and so on, but it does not guarantee that the framework is completely bug-proof. Critical bugs however should be properly fixed else bug fix releases are waranted.
Theia 1.1.0 is released. So shouldn't there be an artefact somewhere that 'records' a working yarn-lock file of that releases working dependencies?
If you want the yarn.lock
file of the 1.1.0 release then you can obtain it from the repo at the 1.1.0 tag. How would you better distribute the yarn.lock
file? It is possible that our yarn.lock
file would interfere with your own product that provides changes/updates. Due to this I believe the best way would be for you to create your own yarn.lock
file for your own application, or use resolutions to use specific dependency versions you want.
My question is, if there is, where do I find it? And if there isn't... then isn't it a bit of a gaping hole in the 'release process' ?
The main project's yarn.lock
is useful to us (project committers) so that we can manage dependencies-related problems caused somewhere in our dependency chain by choosing when we update it - between updates things should remain stable thanks to it.
It can be used as an ad-hoc work-around, to help reduce dependency issues for other Theia applications, but really that's something that any and all serious Theia-based product should do on their own, so they have full control of their own destiny. There are no guarantees when used this way - yarn
will pick between versions of a given dependency according to its own logic, so if you have even a slightly different set of dependencies, it may not help for specific cases.
The main project's yarn.lock
is not an official deliverable of the project.
How would you better distribute the yarn.lock file?
That sounds fair. Yes, I can try if that works for us.
Additionaly, It would be nice if there was 'docker image' representing the release. And that we can assume the yarn.lock file included in that is adequate snapshot representing a 'stable' set of dependencies that is considered as 'working'. (Yes, I know nobody can give 100% guarantees that software is bug free, and its not what I am asking, just that it's intended purpose is that of a 'stable' release, of course there's allways an implied disclaimer that it may not work at all).
so if you have even a slightly different set of dependencies, it may not help for specific cases.
As far as npm dependencies goes, I don't think we'd really want to vary that at all. We are perfectly happy to take on the dependencies that make up a curated full-featured, tested and 'stable' theia instance. We are not going to build that into a larger project by linking it with more npm packages, but rather any customisations will be done by dropping a set of additional vscode extensions of our choice into a plugins folder, nothing more.
So really for this sort of 'use case' we are treating it more as 'black box' with fixed dependencies rather than a dependency (in the npm sense) of a larger npm project.
Perhaps that's not a use case that you really want to explicitly support right now, and its not for me to say that you should. I just hope that you think about it as a valid use case and maybe do something in the way of producing some 'release deliverable' to support this kind of use case better in future.
yarn will pick between versions of a given dependency according to its own logic,
Rigth and that's exactly the problem. It makes it a bit hard to recreate the set of dependencies that you guys already put in the hard work of testing them and are considered as 'working properly'. What we get instead are accidental variations on this tested set and (from my point of view) that results in unnessary breakage.
Don't get me wrong here. We love Theia. It's great product and I gather it will only be getting better.
But it has been quite frustrating for us to deal with the freqeuent breakages which are, for us at least, rather hard to diagnose and explain and result in many hours or in some cases days wasted in trial and errors to try and get something that actually works properly.
You have to understand that we are not explicitly developing extensions for Theia. First and foremost we develop vscode extensions, and that they work in Theia too is a really nice bonus.
Anyway... thanks for all the suggestions. I do have some ideas now on how might try and avoid some of this accidental breakage in the future using yarn.lock from a tagged github release version.
@kdvolder thanks for supplying more details about your experience, even the less positive aspects. It helped me understand your point of view. When dealing with issues in this repo, we do so with a certain angle, which is about "bang-for-the-buck" effort: maintenance of our current example applications and adding some new cool ones that can be maintained with as little effort as possible.
So maintaining an invidual yarn.lock
for each one of them is not in scope for now. Perhaps a little bit down the road, with VS Code extensions support, we can standardize all apps here to use a single Theia application and manage that one a bit more closely, similar to a real product.
OTOH, we're also committers on the upstream Eclipse Theia project, and when I put-on that hat, I see a different angle - many committers also support one or more product based on Theia, some public some internal. Minimising issues that hinder Theia adopters, such as you and ourselves, is of general interest. We've helped each others and others and probably can help you.
First, on the less technical side, I have suggestions. These things take a bit of time investment but may save you from from some of these all-day troubleshooting sessions, or at least share it, troubleshooting in parallel with us.
Taking 10 mins each day or two to browse Theia's spectrum. Major changes that could mess-up your CI are usually announced there. You may also want to have a look at Theia's CHANGELOG before adopting a new release, specially the breaking changes
section. Without your own yarn.lock
, the Theia packages you pull will follow the releases, so you may be adopting new releases sooner than wished.
if something suddenly breaks for you, do a quick search in the eclipse-theia/theia issue tracker for similar and else report it right away there and/or spectrum. Then follow the spectrum thread or issue, help along if you can/want. If the issue is reproduce-able I think we usually have a fix or work-around relatively quick.
Could you clarify which way(s) you consume Theia? Is your application in here: https://github.com/spring-projects/sts4? Are you an extender of one of the docker apps in this repo? Do you consume one or more of the related images we publish to dockerhub under the theiaide
name?
It would be nice if there was 'docker image' representing the release. And that we can assume the yarn.lock file included in that is adequate snapshot representing a 'stable' set of dependencies that is considered as 'working'.
We are perfectly happy to take on the dependencies that make up a curated full-featured, tested and 'stable' theia instance.
It's tempting to think that these desirable qualities can be transmitted by Theia's yarn.lock
. The only situation I would momentarily use the upstream yarn.lock
with a different app, is as a quick troubleshooting "hail Mary"pass in case if may help get over a strange new problem that we're not seeing upstream. I am not sure if that's what you want to do, but pre-preemptively copying Theia's yarn.lock
over to your app's build folder, I think is more likely to destabilise your app's build and robustness rather than help. @akosyakov, @marechal-p WDYT?
As far as npm dependencies goes, I don't think we'd really want to vary that at all.
We are not going to build that into a larger project by linking it with more npm packages, but rather any customisations will be done by dropping a set of additional vscode extensions of our choice into a plugins folder, nothing more.
A simple, vanilla Theia application has a deep dependency tree with 1000+ unique npm dependencies, some of which are present in two or more versions. Having the stars align to keep all these working together is not easy and unpredictable breakage is sure to happen occasionally (it was the case for this issue here).
There are also Theia-level challenges for sure, like avoiding being in a situation where multiple versions of the platform are pulled in one application, making it not work. This I hope we can enhance our tooling to be better able to help/guide the Theia app authors, but it will take time. For example, we recently discussed such things upstream: https://github.com/eclipse-theia/theia/issues/7312
If you maintain your own package.json
that contains Theia extensions and build it into an application, you are currently a Theia application author/maintainer, and that's not always fun. If you are really able to live with a vanilla Theia app, containing only vanilla Theia extensions, that you enhance using VS Code extensions (public ones and potentially your own), then I think you only need to be a little patient - one or a few open-source Theia application products are bound to appear soon, that one can just consume without the headaches of developing and maintaining :)
Perhaps a little bit down the road, with VS Code extensions support, we can standardize all apps here to use a single Theia application and manage that one a bit more closely, similar to a real product.
Looking forward to that :-)
Could you clarify which way(s) you consume Theia? Is your application in here: https://github.com/spring-projects/sts4?
One of the ways we consume it currently is... we build 'snapshot' docker images in our CI build. The way we build them is not 'extending' your package.json / docker build but it greatly resembles it (because we copied a lot from your examples).
See here: https://github.com/spring-projects/sts4/tree/master/concourse/theia-docker-image
The purpose of these CI builds is to allow us to relatively easily try out whether our vscode extensions are working properly in Theia.
We don't really have a 'Theia based product' at the moment. Our deliverables are an Eclipse-based distribution and set of vscode extensions. What we do have is the intention / desire that our vscode extension would also allways be working well and tested with Theia. We are willing to put a moderate amount of effort into trying to ensure this. Usually... it means we grab our latest docker image and do some testing just prior to releasing our vscode extensions.
Are you an extender of one of the docker apps in this repo? Do you consume one or more of the related images we publish to dockerhub under the theiaide name?
Normally, no, we don't use the images from dockerhub. However in the last weeks we have been experimenting with 'eduk8s' (https://github.com/eduk8s) and trying to create a 'workshop docker image) that includes Theia enhanced with our STS 4 plugins and a few other interesting third-party vscode plugins in the 'k8s' space.
This is where the latest 'long trial and error' happened. Finally settled on something that doesn't try to build its own theia anymore (because it wasn't working) but instead copied theia built artefacts from one of your older docker images.
See here:
https://github.com/kdvolder/lab-spring-boot/blob/master/Dockerfile#L21
I don't think about these images as products, but examples. It is helpful that we don't have yarn.lock here, since we get reports if somethings gets broken, can fix it and communicate to adopters.
Hi @kdvolder
The purpose of these CI builds is to allow us to relatively easily try out whether our vscode extensions are working properly in Theia.
We don't really have a 'Theia based product' at the moment. Our deliverables are an Eclipse-based distribution and set of vscode extensions. What we do have is the intention / desire that our vscode extension would also allways be working well and tested with Theia. We are willing to put a moderate amount of effort into trying to ensure this. Usually... it means we grab our latest docker image and do some testing just prior to releasing our vscode extensions.
Thank you fir the extra context- I understand further what you're doing with Theia and I can see why, at this time, it may not be worth it for you to go "full in".
Feel free to ping us on GitHub or Spectrum next time you have Theia problems and hopefully we can make it less painful.
Finally settled on something that doesn't try to build its own theia anymore (because it wasn't working) but instead copied theia built artefacts from one of your older docker images.
Ah, this is pretty much what I wanted to suggest as a work-around. The problem you opened this issue-for initially went un-detected during CI and the images were published with the problem for a little while. But usually most big problems are caught in CI, and so you could pick the last published image of a given type and use the contained Theia application successfully, probably most of the time. Else you can go back in history and pick an image that maps to the previous Theia version, as you did here.
Since a few days we seem to be having problems with our STS4 theia docker images. Several things are going wrong. Perhaps most notable is that there are no menus at the top of the window where they are supposed to be.
Also trying to use 'CTRL-SHIFT-P' to open the 'command quick search' doesn't work (instead it opens the browser's 'print page' dialog).
We went back to 'the source' that our images are based on (i.e. the 'official' theia-java' docker image) and it exhibits the same problem.
Please find below a 'dump' of errors / output from the browser js console.