ST6RI-753 Move to Eclipse version 2024-03

[TheKorpos]

Major highlighs

• Moved to Eclipse 2024-03
• Compiling all projects using Java 17
• Updated Tycho to 4.0.7
• Updated Xtext to 2.34.0
• Updated Xpect to 0.3.0
• Added target definition
• Updated Oomph setup
• Oomph setup activates the target definition
• Updated maven wrapper to 3.9.6

jupyter-sysml-kernel

Thejupyter-sysml-kernel project uses gradle and a gradle plugin to build the kernel. This plugin only works with older versions of gradle that do not support Java 17. The latest update for this plugin that works with gradle 8+ hasn't been deployed to any remote plugin sites yet. As a workaround we decided to build our own .jar and use plugin locally.

• Updated gradle to 8.0
• Updated the sysml-kernel-installer gradle plugin to 3.0.0-SNAPSHOT
• Added temporary workaround in the root pom to address issue

[himi]

As I mentioned in the meeting, I hit the issue of Eclipse 2024-03 on MacOS 14.3.1 (Sonoma). See below for details: https://github.com/eclipse-platform/eclipse.platform.swt/issues/1012

I needed to upgrade the Eclipse to the prerelease of 2024-06, 4.32M1.

[himi]

I confirmed it worked. I think we should restore tycho-pomless to tycho-build 4.0.7 aligning to the base tycho version. Thank you for the tough work for solving many intricate issues.

[himi]

I pushed d12f6e0 to move buildscript section to build.gradle. Also added eclipse plugin because it's annoying to resolve classpath in Eclipse. Please tell me if you find any issues on this change.

[himi]

~It looked the travis build was failed and I'll investigate it (but it worked in my environment). Sorry.~. It seems just it failed to download yarn for jupyterlab extension build. Please ignore.

[himi]

@seidewitz, may I add .yarn/ to .gitignore? It's annoying if we use yarn3 or later.

[TheKorpos]

Thank you @himi!

I pushed d12f6e0 to move buildscript section to build.gradle. Also added eclipse plugin because it's annoying to resolve classpath in Eclipse. Please tell me if you find any issues on this change.

Yes. This is exacly what I was about to push too

As I mentioned in the meeting, I hit the issue of Eclipse 2024-03 on MacOS 14.3.1 (Sonoma). See below for details: eclipse-platform/eclipse.platform.swt#1012

I needed to upgrade the Eclipse to the prerelease of 2024-06, 4.32M1.

The problem with 2024-06 that it is java 21+ and it isn't out yet. Since we have a target definition now it should be esier to jump between versions if needed in your dev environment. We just need to be careful what dependencies we use.

[himi]

The problem with 2024-06 that it is java 21+ and it isn't out yet. Since we have a target definition now it should be esier to jump between versions if needed in your dev environment. We just need to be careful what dependencies we use.

Indeed. I needed to use that version just to avoid this critical bug. Let's discuss on it later. It is a help for poor MacOS Sonoma users like me.

[seidewitz]

It seems just it failed to download yarn for jupyterlab extension build. Please ignore.

I am also having the download of yarn fail for my local build, with a 139 error. How do I fix that?

[himi]

Can you show your build log? I just restarted the build. If I remember correctly, it just failed DNS name lookup but the travis log seems to be lost.

[himi]

This worked on Windows 10 and 11, and Linux environment (due to the update of Gradle, we needed to modify our docker environment). However, I believe we should restore the exclude of kernelResources configuration, which prevents the inclusion of jupyter management files if you do not have any specific reasons.

[seidewitz]

@himi The Travis builds seem to be working now. But I can't build it on my local machine. Here is the error I am getting:

[INFO] --- frontend:1.12.0:install-node-and-yarn (install-node-and-yarn) @ jupyterlab-sysml ---
[INFO] Installing node version v16.2.0
[INFO] Unpacking /Users/seidewitz/.m2/repository/com/github/eirslett/node/16.2.0/node-16.2.0-darwin-arm64.tar.gz into /Users/seidewitz/Documents/Work/git/SysML-v2-Pilot-Implementation/org.omg.sysml.jupyter.jupyterlab/target/node/tmp
[INFO] Copying node binary from /Users/seidewitz/Documents/Work/git/SysML-v2-Pilot-Implementation/org.omg.sysml.jupyter.jupyterlab/target/node/tmp/node-v16.2.0-darwin-arm64/bin/node to /Users/seidewitz/Documents/Work/git/SysML-v2-Pilot-Implementation/org.omg.sysml.jupyter.jupyterlab/target/node/node
[INFO] Installed node locally.
[INFO] Installing Yarn version v1.22.19
[INFO] Unpacking /Users/seidewitz/.m2/repository/com/github/eirslett/yarn/1.22.19/yarn-1.22.19.tar.gz into /Users/seidewitz/Documents/Work/git/SysML-v2-Pilot-Implementation/org.omg.sysml.jupyter.jupyterlab/target/node/yarn
[INFO] Installed Yarn locally.
[INFO] 
[INFO] --- frontend:1.12.0:yarn (yarn-install) @ jupyterlab-sysml ---
[INFO] Running 'yarn install' in /Users/seidewitz/Documents/Work/git/SysML-v2-Pilot-Implementation/org.omg.sysml.jupyter.jupyterlab
...
[ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.12.0:yarn (yarn-install) on project 
jupyterlab-sysml: Failed to run task: 'yarn install' failed. org.apache.commons.exec.ExecuteException: 
Process exited with an error: 139 (Exit value: 139) -> [Help 1]

A 139 error code is a segment fault. yarn install runs successfully for me from the command line.

[himi]

Hmmm. It seemed nodejs crashed in your environment. ~If you already installed yarn and nodejs in your environment, try yarn install in org.omg.sysml.jupyter.jupyterlab directory. If you do not have them installed,~ please try to clean /Users/seidewitz/.m2 and mvn package again. If error persist, maybe your MacOS environment has a problem to execute node-16.2.0, and please let me know.

BTW, what are the versions of yarn and nodejs in your local environment? More specifically, what are the results of yarn --version and node --version?

Just FYI, below is the log message in my MacOS environment:

[INFO] --- frontend:1.12.0:install-node-and-yarn (install-node-and-yarn) @ jupyterlab-sysml ---
[INFO] Node v16.2.0 is already installed.
[INFO] Yarn 1.22.19 is already installed.
[INFO] 
[INFO] --- frontend:1.12.0:yarn (yarn-install) @ jupyterlab-sysml ---
[INFO] Running 'yarn install' in /Users/himi/git/SysML-v2-Pilot-Implementation/org.omg.sysml.jupyter.jupyterlab
[INFO] yarn install v1.22.19
[INFO] [1/4] Resolving packages...
[INFO] success Already up-to-date.
[INFO] Done in 0.08s.

[seidewitz]

Clearing .m2 didn't fix the problem.

I have Yarn 1.22.19, but Node 15.14.0. I didn't think Jupyter 2.x worked with Node 16, which is why we were using 15. The install.sh script for Jupyter in org.omg.sysml.jupyter.installer on this branch still has nodejs=15.* in the conda install line.

[himi]

So please try to change node.version in pom.xml to 15.14.0

I suspect your node is intel binary. What is the result of

file `which node`

[seidewitz]

OK, that worked. However, I checked, and the node version in the POM was previously 16.2.0. In fact, we changed to node 16 in the POM back in 2021, even though we have been installing Jupyter using an older version. So why is the build not working with 16 now?

[himi]

node 15 does not have arm64 binary. That's why we upgraded to 16. And the build requires only yarn and does not require jupyter. For this reason, I do not want to downgrade node. Can you try node 16.20.2? The latest version of node v16.

[seidewitz]

Yes, node 15.14.0 has an arm64 binary (which is what I have installed). That is why, in release 2023-11, we moved from nodejs=14.* to 15.* in the Jupyter install script for Unix/MacOS, because it would run on arm64 and still worked with Jupyter 2.

[himi]

Hmmm.. I could not find darwin-arm64 binary in https://nodejs.org/download/release/v15.14.0/. How did you get it?

[seidewitz]

https://nodejs.org/download/release/v15.14.0/node-v15.14.0.pkg

At least, I just tried that, and it does install an arm64 version. I can't remember if that is exactly how I did it before. (Actually, I probably did conda install nodejs="15.*", which also gives 15.14.0 for arm64.)

That's also what you get if you select 15.4.0 and ARM64 at https://nodejs.org/en/download.

[himi]

I did not notice MacOS pkg is ready for 15.14.0. However maven cannot automatically download and install it locally. So in my MacOS, maven downloaded x64 binary, and it could work by using Rosetta. Do you want to downgrade to 15.14.0 or try to upgrade to 16.20.2, or upgrade your MacOS version if this issue comes from your specific old MacOS version.

[seidewitz]

Interestingly, conda seems able to get the 15.14.0 version for ARM.

In any case, I updated to 16.13.1 (because that's what conda grabbed), but the build still failed with node.version=16.2.0. However, if I set node.version to 16.13.1 or 16.20.2, then it worked.

The question is, why did it work before in the older version of Eclipse with node.version=16.2.0, when I had 15.14.0 installed?

[himi]

The first question is easy to answer. conda has its own package system and possibly they made their package from the distribution.

The second question is quite interesting. Maven uses JVM which actually launches node. Maybe this issue comes from the combination of JVM version and node version. But that might happen!

So how about going with 16.20.2? I think we should use the latest of v16.

[seidewitz]

I have confirmed that I can rebuild the Jupyter kernel and run Jupyter after a clean build with node.version=16.20.2. However, if I run the release install script for Jupyter, then the node installation drops back to 15.4.0 ... and yet the build still runs successfully!

So it's all a bit confusing...

Clearly, we also need to move to Jupyter 3. using nodejs 16.. But maybe we can get this PR closed first.

[himi]

conda is quite useful if it works well. But once a problem happens, it makes situations quite complicated and you will be put in the muddy. I hope to move to jupyterlab 4 because the extension mechanism is much cleaner.

[seidewitz]

I went ahead and changed node.version in the top-level POM to 16.20.2.

@TheKorpos Please confirm that you are OK with this and any other changes that have been made.

[TheKorpos]

@seidewitz, @himi Thank you for the testing and the fixes.

I went ahead and changed node.version in the top-level POM to 16.20.2.

@TheKorpos Please confirm that you are OK with this and any other changes that have been made.

The changes look fine. I performed a last testing trip and everything seems to work as intended.

[himi]

I have two requests to be changed as I mentioned.

1. Restore the setting of excludes of kernelResource
2. Add .yarn/ to .gitignore

@seidewitz and @TheKorpos, If you are ok, I'll make these changes. Your explanation is appreciated If you reject them.

[seidewitz]

@TheKorpos should address your request on kernelResource. I am trying to find were .yarn even is right now.

[himi]

.yarn/ subfolder is not used by Yarn1.x. I think it's introduced by Yarn 3.x or later. I need to use several version of yarn to test and it's annoying with .yarn folder. If you do not want to add it, I will bear it because I need to use many environments.

[TheKorpos]

@himi I've restored the exclusion. This is what you meant right?

[himi]

@TheKorpos thank you. It worked at least in my MacOS. I will check it in all of my environment after the response from @seidewitz.

[seidewitz]

@himi I think it is kind of odd to add something to .gitignore for a capability that we are not actually using in our current build. There is nothing to keep you from updating your .gitignore locally for this without committing it to the repository.

[himi]

@seidewitz. I see. So please go ahead.