devonfw / ide

Tool to automate setup and update of development environment (esp. for Java projects).
Apache License 2.0
33 stars 101 forks source link

checksum verification for custom software repository #1276

Closed hohwille closed 1 year ago

hohwille commented 1 year ago

We have a problem when using the custom software repository feature as the checksum verification is implemented without considering this feature:

*** Setting up tomcat ***
Success: run command mkdir
Starting installation of tomcat in version 9.0.65 to /d/projects/test/software/tomcat
Trying to download tomcat-9.0.65.tgz from http://custom.repository.com/software-repository/project//tomcat/9.0.65/tomcat-9.0.65.tgz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 15.6M  100 15.6M    0     0  1041k      0  0:00:15  0:00:15 --:--:-- 1653k
Download of tomcat-9.0.65.tgz from http://custom.repository.com/software-repository/project//tomcat/9.0.65/tomcat-9.0.65.tgz succeeded.
cat: /d/projects/test/urls/tomcat//9.0.65/-_.urls.sha256: No such file or directory
sha256sum: /c/Users/hohwille/Downloads/devonfw-ide/custom.repository.com/_software-repository_project_/: Is a directory
Checksum  is correct
Success: run command tar
Successfully extracted archive tomcat-9.0.65.tgz to /d/projects/test/updates/extracted/tomcat
Success: run command mv
Successfully installed tomcat
The software tomcat has been added. You need to rerun 'devon' command without arguments or restart your terminal to update your PATH so the newly installed software will be found.

So lets break this down:

Trying to download tomcat-9.0.65.tgz from http://custom.repository.com/software-repository/project//tomcat/9.0.65/tomcat-9.0.65.tgz

double slash (//) is not so nice but rather unimportant.

cat: /d/projects/test/urls/tomcat//9.0.65/-_.urls.sha256: No such file or directory

This is the actual error: In case of a software repo, we can not use the official metadata from ide-urls. Even worse in this case, if it would have computed the proper path correctly, it would have aborted the installation due to a checksum mismatch. If we want support for checksum verification from a custom software repository, we need to download the expected checksum from that repository rather than taking it from our urls git repo what is wrong.

I assume that the operating system is - here what usually means that we do not distinguish between OSes as we have a portable binary. However, the logic to compute the checksum file used this as a real OS and computed the filename -_.urls.sha256 what is also another bug.

sha256sum: /c/Users/hohwille/Downloads/devonfw-ide/custom.repository.com/_software-repositoryproject/: Is a directory

This is another interesting bug. We did not pass the full path to the downloaded file to sha256sum but instead the parent directory. I do not yet understand why this was happening.

Checksum is correct

This is a lie. When there is no checksum, it is not correct.

Expected result: For the simple bugfix, I would expect our checksum verification to skip the check in case of a software repo. On the longer run, we can add a feature to build a potential download URL to an according checksum file (same URL as the download file with .sha256 appended) and trying to download that as well. If this fails (with 404), we need to proceed without error. If we get valid content (200), we should validate the checksum.