#1084 Adapt scripts to use ide-urls

[CREITZ25]

1084 Adapt scripts to use ide-urls

[hohwille]

I started to test this feature branch locally:

hohwille@CE49454 MSYS /d/projects/devonfw
$ devon ide update

*** Setting up az ***
Success: run command mkdir
Starting installation of az in version 2.46.0 to /d/projects/devonfw/software/az
Trying to download az-2.46.0-windows.msi from https://azcliprod.blob.core.windows.net/msi/azure-cli-2.46.0.msi
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 55.2M  100 55.2M    0     0  3880k      0  0:00:14  0:00:14 --:--:-- 4909k
Download of az-2.46.0-windows.msi from https://azcliprod.blob.core.windows.net/msi/azure-cli-2.46.0.msi succeeded.
Successfully extracted archive az-2.46.0-windows.msi to /d/projects/devonfw/updates/extracted/az
Creating backup by moving existing /d/projects/devonfw/software/az to /d/projects/devonfw/updates/backups/23-06-23
Success: run command mv
Successfully installed az
The software az has been added. You need to rerun 'devon' command without arguments or restart your terminal to update your PATH so the newly installed software will be found.
Junction created for D:\projects\devonfw\software\az\bin <<===>> D:\projects\devonfw\software\az\wbin
Success: run command cmd
azure-cli                         2.46.0 *

core                              2.46.0 *
telemetry                          1.0.8

Dependencies:
msal                              1.20.0
azure-mgmt-resource             21.1.0b1

Python location 'D:\projects\devonfw\software\az\python.exe'
Extensions directory 'D:/projects/devonfw/conf/.azure\cliextensions'

Python (Windows) 3.10.10 (tags/v3.10.10:aad5f6a, Feb  7 2023, 17:05:00) [MSC v.1929 32 bit (Intel)]

Legal docs and information: aka.ms/AzureCliLegal

You have 2 update(s) available. Consider updating your CLI installation with 'az upgrade'

[Survey] Help us improve Azure CLI by sharing your experience. This survey should take about 5 minutes. Run 'az survey' to open in browser. Learn more at https://go.microsoft.com/fwlink/?linkid=2203309

Success: verify installation of az (/d/projects/devonfw/software/az/bin/az)

*** Setting up dotnet ***
Updating dotnet from version 6.0.403 to version 6.0.406...
Success: run command mkdir
Starting installation of dotnet in version 6.0.406 to /d/projects/devonfw/software/dotnet
Trying to download dotnet-6.0.406-windows.zip from https://dotnetcli.azureedge.net/dotnet/Sdk/6.0.406/dotnet-sdk-6.0.406-win-x64.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  250M  100  250M    0     0  3891k      0  0:01:05  0:01:05 --:--:-- 6350k
Download of dotnet-6.0.406-windows.zip from https://dotnetcli.azureedge.net/dotnet/Sdk/6.0.406/dotnet-sdk-6.0.406-win-x64.zip succeeded.
Success: run command unzip
Successfully extracted archive dotnet-6.0.406-windows.zip to /d/projects/devonfw/updates/extracted/dotnet
Creating backup by moving existing /d/projects/devonfw/software/dotnet to /d/projects/devonfw/updates/backups/23-06-23
Success: run command mv
Successfully installed dotnet
The software dotnet has been added. You need to rerun 'devon' command without arguments or restart your terminal to update your PATH so the newly installed software will be found.
6.0.406
Success: verify installation of dotnet (dotnet)

*** Setting up eclipse ***
Version 2023-03 of eclipse is already installed at /d/projects/devonfw/software/eclipse.
name=Eclipse Platform
id=org.eclipse.platform
version=4.26.0
Success: verify installation of eclipse (cat)

*** Setting up intellij ***
Version 2023.1 of intellij is already installed at /d/projects/devonfw/software/intellij.
{
  "name": "IntelliJ IDEA",
  "version": "2023.1",
  "buildNumber": "231.8109.175",
  "productCode": "IC",
  "dataDirectoryName": "IdeaIC2023.1",
Success: verify installation of intellij (head)

*** Setting up java ***
Version 11.0.17_8 of java is already installed at /d/projects/devonfw/software/java.
openjdk version "11.0.17" 2022-10-18
OpenJDK Runtime Environment Temurin-11.0.17+8 (build 11.0.17+8)
OpenJDK 64-Bit Server VM Temurin-11.0.17+8 (build 11.0.17+8, mixed mode)
Success: verify installation of java (java)
Version 17.0.5_8 of java is already installed at /d/projects/devonfw/software/extra/java.
openjdk version "11.0.17" 2022-10-18
OpenJDK Runtime Environment Temurin-11.0.17+8 (build 11.0.17+8)
OpenJDK 64-Bit Server VM Temurin-11.0.17+8 (build 11.0.17+8, mixed mode)
Success: verify installation of java (java)

*** Setting up mvn ***
Updating mvn from version 3.8.6 to version 3.9.0...
Success: run command mkdir
Starting installation of mvn in version 3.9.0 to /d/projects/devonfw/software/maven
Trying to download mvn-3.9.0.tar.gz from https://archive.apache.org/dist/maven/maven-3/3.9.0/binaries/apache-maven-3.9.0-bin.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 8812k  100 8812k    0     0  3403k      0  0:00:02  0:00:02 --:--:-- 3405k
Download of mvn-3.9.0.tar.gz from https://archive.apache.org/dist/maven/maven-3/3.9.0/binaries/apache-maven-3.9.0-bin.tar.gz succeeded.
Success: run command tar
Successfully extracted archive mvn-3.9.0.tar.gz to /d/projects/devonfw/updates/extracted/maven
Creating backup by moving existing /d/projects/devonfw/software/maven to /d/projects/devonfw/updates/backups/23-06-23
Success: run command mv
Successfully installed mvn
The software maven has been added. You need to rerun 'devon' command without arguments or restart your terminal to update your PATH so the newly installed software will be found.
/d/projects/devonfw/scripts/command/../functions: line 84: /d/projects/devonfw/software/mvn/bin/mvn: No such file or directory

******** ATTENTION ********
Failed to verify installation of mvn (/d/projects/devonfw/software/mvn/bin/mvn)
We are sorry for the inconvenience. Please check the above errors, resolve them and try again.
Exit code was 127
Command 'mvn setup' failed with exit code 127

******** ATTENTION ********
Failed to run devon mvn setup
We are sorry for the inconvenience. Please check the above errors, resolve them and try again.
Exit code was 127

hohwille@CE49454 MSYS /d/projects/devonfw
$ ls -la software/maven/bin/mvn
-rwxr-xr-x 1 hohwille 1049089 5883 Jan 31 10:57 software/maven/bin/mvn*

I do not understand the error. All seems to be installed correctly. For the record: I still do not have urls folder/repo in my IDE. Maybe the error I got is not related to the changes of this PR at all.

[hohwille]

hohwille@CE49454 MSYS /d/projects/devonfw
$ az -v
azure-cli                         2.46.0 *

core                              2.46.0 *
telemetry                          1.0.8

Dependencies:
msal                              1.20.0
azure-mgmt-resource             21.1.0b1

Python location 'D:\projects\devonfw\software\az\python.exe'
Extensions directory 'D:/projects/devonfw/conf/.azure\cliextensions'

Python (Windows) 3.10.10 (tags/v3.10.10:aad5f6a, Feb  7 2023, 17:05:00) [MSC v.1929 32 bit (Intel)]

Legal docs and information: aka.ms/AzureCliLegal

You have 2 update(s) available. Consider updating your CLI installation with 'az upgrade'

hohwille@CE49454 MSYS /d/projects/devonfw
$ devon az version set latest
AZ_VERSION=2.46.0 has been set in /d/projects/devonfw/settings/devon.properties
To install that version call the following command:
devon az setup

hohwille@CE49454 MSYS /d/projects/devonfw
$ rm -rf mirrors/

hohwille@CE49454 MSYS /d/projects/devonfw
$ devon az version set latest
Cloning git repositroy from https://github.com/devonfw/ide-mirrors.git
Cloning into '.'...
remote: Enumerating objects: 882, done.
remote: Counting objects: 100% (223/223), done.
remote: Compressing objects: 100% (120/120), done.
Receiving objects: 100% (882/882), 120.61 KiB | 1.61 MiB/s, done.

Resolving deltas: 100% (431/431), done.
Success: run command git
AZ_VERSION=2.46.0 has been set in /d/projects/devonfw/settings/devon.properties
To install that version call the following command:
devon az setup

hohwille@CE49454 MSYS /d/projects/devonfw
$ ls urls
ls: cannot access 'urls': No such file or directory

May it is better to drop mirrors support and avoid the complexity of maintaining both variants at the same time. I will test again with manual cloning of urls however the idea was that if neither urls nor mirrors is present that then urls is created.

[hohwille]

OK, I found the maven problem:

$ devon mvn setup
target_path=/d/projects/devonfw/software/java
target_path=/d/projects/devonfw/software/extra/java
doInstall mvn    /d/projects/devonfw/software/maven -
target_path=/d/projects/devonfw/software/maven
Version 3.9.2 of mvn is already installed at /d/projects/devonfw/software/maven.
/d/projects/devonfw/scripts/command/../functions: line 84: /d/projects/devonfw/software/mvn/bin/mvn: No such file or directory

My scripts/devon.properties file was not symlinked to the one from git repo. My mistake.

[hohwille]

Next bug is the migration script (due to other PRs). I fixed it locally.

[hohwille]

Next bug is MAVEN_HOME vs. MVN_HOME inconsistency: https://github.com/devonfw/ide/blob/4b46ec935dd8fdb33b9f3fc72e8264c4340abae0/scripts/src/main/resources/scripts/devon.properties#L18 https://github.com/devonfw/ide/blob/4b46ec935dd8fdb33b9f3fc72e8264c4340abae0/scripts/src/main/resources/scripts/command/mvn#L15 BUT https://github.com/devonfw/ide/blob/4b46ec935dd8fdb33b9f3fc72e8264c4340abae0/scripts/src/main/resources/scripts/migrations/2023.06.001#L9

[hohwille]

devon «tool» version set [Tab] does not work with ide-urls. I will fix it.

[hohwille]

$ devon gcviewer setup
Success: run command mkdir
Starting installation of gcviewer in version 1.36 to /d/projects/devonfw/software/gcviewer
Trying to download gcviewer-1.36-gcviewer-windows.jar from https://sourceforge.net/projects/gcviewer/files/gcviewer-1.36.jar
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   718    0   718    0     0   1928      0 --:--:-- --:--:-- --:--:--  1940
100   734    0   734    0     0   1200      0 --:--:-- --:--:-- --:--:--  1200
100   971    0   971    0     0   1055      0 --:--:-- --:--:-- --:--:--  1055
  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0
curl: (35) schannel: next InitializeSecurityContext failed: Unknown error (0x80092012) - The revocation function was unable to check revocation for the certificate.
Failed to download https://sourceforge.net/projects/gcviewer/files/gcviewer-1.36.jar with exit code 35

[hohwille]

The error is reproducible but also happens with the latest published stable release of devonfw-ide so no bug of changes for ide-urls.

[hohwille]

To get closer towards our release, I decided to "fork" your PR and derive my own PR from your changes. Thanks @CREITZ25 for your excellent work 👍 This PR is now replaced by #1208