Dojo day 2 - Windows WSL2 - docs, performance, recommendations

[planetf1]

Windows users need to use WSL2 for our Dojos, and explicitly day 2 where we are coding using IntelliJ

We currently reference WSL2, but it was clear in the dojo that more guidance on using Windows+WSL2 would be desirable. This should also extend more generally to our snippets relating to build, edit etc, and for the dojo, use of kafka and zookeeper

I tried with WSL2 after the meeting

• Windows 11
• Ubuntu 20.04
• Maven, java (adoptium - 17 in my case) already installed in Ubuntu ie everything to do a command line build
• WSL2
• egeria, egeria-dev-projects, egeria-samples already downloaded on Ubuntu
• Clean install of intelliJ (ultimate)
• Clicked ‘new project’ on wizard that popped up
• Navigated into the source tree under //$wsl
• Intelli/J seemed to auto-detect the right jdk etc
• Maven was picked up automatically (there is a setting for choosing jdk. and the wsl jdk is applied automatically by one of the tasks that runs after opening)
• Clicked option to skip gradle configuration (just choose one or another)
• Left IDE to sort itself out resolving dependencies
• All projects show green
• Terminal in intellij correctly opens up directly with the Ubuntu runtime
• The maven window also works fine, so I can, for example, build the top level project (with -Pquick -DskipFVT). (you will see some yellow warnings - these are just output by delombok cc: @Mandy Chessell )
• The platform launches fine. Everything in the Command line seen in the run window is Ubuntu-centric . No references to windows paths

We’ll need to have a slightly more controlled configuration with screenshots, and run through the entire session, capture version numbers etc, but at least with current sw there does seem a fairly smooth path - so far. I’ve also not progressed to running kafka etc.

It would be interesting to compare with other’s experiences. It’s possibly related to using a fairly current version of IntelliJ as this is a relatively new area of support — but the above is promising, and looks like a nice development environment.

https://www.jetbrains.com/help/idea/how-to-use-wsl-development-environment-in-product.html#create_project_for_wsl

If not using IntelliJ, remember to do all edits natively in WSL2, or use unix line endings if using a windows based editor.

[planetf1]

In a rerun of the Dojo, 2 attendees had multiple problems running IntelliJ under WSL2 for building Egeria

For egeria itself, we can avoid this by building egeria from the command line.

However later tutorials - and developers working on egeria more generally, would like to build on Windows.

Issue seen today is that when importing/setting up a new WSL-2 based egeria project for the first time, the import & downloading of dependencies + code indexing is taking over an hour to complete. During this time the IDE is unresponsive. If a new project is created, the UI never appears. This was with 2021.3

https://youtrack.jetbrains.com/issue/IDEA-255733 addresses slow indexing. In part this was addressed by 2021.2, but unclear how much we can expect performance to approach a fully native solution.

Other options for windows may include

• reverting to supporting windows 'natively' - fixing issues where windows is not working properly, so that build etc can be local (but this could lead to ongoing issues? support?)

  • advise that everything is run in the linux environment, including the linux version of the IDE - or run in a VM

  • trying an alternate IDE by Visual Studio code - however this is then very different to what all our devs run.

[planetf1]

Attempting to run IntelliJ IDEA natively on windows can also fail:

See references to https://intellij-support.jetbrains.com/hc/en-us/community/posts/115000759004-Unable-to-start-correctly-0xc000007b - where it seems idea64.exe fails if ASLR protection is enable

However in my case the security settings did not help, though running idea.bat directly did work (Another tip for the notes)

[planetf1]

Attempt to clone into windows directly (win11) resulted in:

2 [main] sh (1224) C:\Program Files\Git\usr\bin\sh.exe: *** fatal error - cygheap base mismatch detected - 0x1329408/0x13E9408. This problem is probably due to using incompatible versions of the cygwin DLL. Search for cygwin1.dll using the Windows Start->Find/Search facility and delete all but the most recent version. The most recent version *should* reside in x:\cygwin\bin, where 'x' is the drive on which you have installed the cygwin distribution. Rebooting is also suggested if you are unable to find another cygwin DLL. 0 [main] sh 335 dofork: child -1 - forked process 1224 died unexpectedly, retry 0, exit code 0xC0000142, errno 11 C:/Program Files/Git/mingw64/libexec/git-core\git-submodule: fork: retry: Resource temporarily unavailable 5 [main] sh (7260) C:\Program Files\Git\usr\bin\sh.exe: *** fatal error - cygheap base mismatch detected - 0x1329408/0x12B9408. This problem is probably due to using incompatible versions of the cygwin DLL. Search for cygwin1.dll using the Windows Start->Find/Search facility and delete all but the most recent version. The most recent version *should* reside in x:\cygwin\bin, where 'x' is the drive on which you have installed the cygwin distribution. Rebooting is also suggested if you are unable to find another cygwin DLL. 1139854 [main] sh 335 dofork: child -1 - forked process 7260 died unexpectedly, retry 0, exit code 0xC0000142, errno 11 C:/Program Files/Git/mingw64/libexec/git-core\git-submodule: fork: retry: Resource temporarily unavailable 3 [main] sh (3500) C:\Program Files\Git\usr\bin\sh.exe: *** fatal error - cygheap base mismatch detected - 0x1329408/0x13F9408. This problem is probably due to using incompatible versions of the cygwin DLL. Search for cygwin1.dll using the Windows Start->Find/Search facility and delete all but the most recent version. The most recent version *should* reside in x:\cygwin\bin, where 'x' is the drive on which you have installed the cygwin distribution. Rebooting is also suggested if you are unable to find another cygwin DLL. 3252569 [main] sh 335 dofork: child -1 - forked process 3500 died unexpectedly, retry 0, exit code 0xC0000142, errno 11 C:/Program Files/Git/mingw64/libexec/git-core\git-submodule: fork: retry: Resource temporarily unavailable 4 [main] sh (6640) C:\Program Files\Git\usr\bin\sh.exe: *** fatal error - cygheap base mismatch detected - 0x1329408/0x1259408. This problem is probably due to using incompatible versions of the cygwin DLL. Search for cygwin1.dll using the Windows Start->Find/Search facility and delete all but the most recent version. The most recent version *should* reside in x:\cygwin\bin, where 'x' is the drive on which you have installed the cygwin distribution. Rebooting is also suggested if you are unable to find another cygwin DLL. 7380294 [main] sh 335 dofork: child -1 - forked process 6640 died unexpectedly, retry 0, exit code 0xC0000142, errno 11 C:/Program Files/Git/mingw64/libexec/git-core\git-submodule: fork: retry: Resource temporarily unavailable 6 [main] sh (8336) C:\Program Files\Git\usr\bin\sh.exe: *** fatal error - cygheap base mismatch detected - 0x1329408/0x1379408. This problem is probably due to using incompatible versions of the cygwin DLL. Search for cygwin1.dll using the Windows Start->Find/Search facility and delete all but the most recent version. The most recent version *should* reside in x:\cygwin\bin, where 'x' is the drive on which you have installed the cygwin distribution. Rebooting is also suggested if you are unable to find another cygwin DLL. 15543909 [main] sh 335 dofork: child -1 - forked process 8336 died unexpectedly, retry 0, exit code 0xC0000142, errno 11 C:/Program Files/Git/mingw64/libexec/git-core\git-submodule: fork: Resource temporarily unavailable

Cloning at the cli (with long paths enabled) then importing works, and only takes a little longer than macOS - perhaps 6 minutes.

However maven project import is flaky, failing several times with unresolved dependencies -- which do exist. (network issues?). Also intermittent warnings 'Clean up the broken artifacts data (.lastUpdated files) and reload the project.` Also had issues with import natively without 3GB RAM allocated

[planetf1]

Note - Win native java build will fail. Currently FVTs fail, due to launching the 'java' process vs 'java.exe' - could be addressed by refining how the file is determine, or using a diff plugin/javaexec directly. not opening issue yet until we determine best approach overall for windows

[planetf1]

There is another possible option

The latest version of Jetbrains IntelliJ offers 'JetBrains gateway' as a standalone package, or via File->Remote Development. This requires SSH connectivity to the host, runs the IDE remotely in 'headless' mode, but runs the GUI locally. This may work differently to WSL2 integration - unknown.

[adinhtdsibm]

@planetf1 Do you want me to try any setup on my system? I use Windows 10 and if there are tasks that I need to perform, I can give a try.

[planetf1]

That would be super thanks.

I believe windows 10 has evolved WSL support in a few iterations. Most recently WSL2 was added -- we probably need docs around how to ensure that is being used (vs WSL version 1), and that kernel updates are done.

I think WSLg (which would allow everything to run in wsl including the UI) requires WIndows 11.

So that probably means the best option - is to try the built in support for WSL2 in IntelliJ? We don't have docs/screenshots, though there are intellij posts on the subject. When we tried in the dojo we found the performance problematic - after importing egeria, indexing seemed to take up to an hour, and the UI was unresponsive during this time?

If you have other proposals please do suggest :-)

[planetf1]

I've had another try today on a standalone windows 11 system with WSL2 & IntelliJ 2022.1 Community Edition

I did the inital setup (git clone, installing compiler, maven etc) entirely from within Ubuntu I then did a File->Open on the //wsl$/....etc.... directory

Indexing is a little slow, perhaps taking 10-20 mins, but the IDE remained usable

If a maven project is detected, the UI didn't become unresponsive, but tasks to import maven plugins never completed even after several hours . I attempted this twice.

However using 'gradle' the project setup took 1-2 minutes, and also gradle tasks can be run - such as gradle build -x test -x javadoc and again take seconds/minutes - works very well

To remove maven setup one can go into the maven tool window & remove the project To enable gradle one can right click on 'build.gradle' in the root and 'add as gradle project'

In summary, at this point, maven usage seems unrealistic for this configuration. Gradle works well, though it should be noted the final archive created is in a different location, and also maven artifacts are not published into local .m2. Still it's probably the most viable for development on windows platform.

To use maven, just using the CLI is probably the better approach

My tests were run on old hw running beta software, so may not totally reproduce what others see If any windows oriented developers could check this out, and possibly create detailed instructions that would be useful

[planetf1]

Added some further docs on using WIndows to build Egeria note - this isn't fully linked into all the material, but is focussed on the task 'building egeria source'. We need more engagement from windows developers to update the more general flow once we are happy with the approach. For now this update at least captures some of the observations made here, and offers some ideas of what could work - as well as provide a link back here for further discussion & help. cc: @mandy-chessell