next.openhab.org does not reflect current state of addons repo

[holgerfriedrich]

Somehow https://next.openhab.org/addons/ seems out of date....

When I select the 4.0.0: For example the following PRs are not included, openhab/openhab-addons#14983 (misses newly added channel targetenergy) openhab/openhab-addons#15012 (still shows luftdaten instead of sensor.community) openhab/openhab-addons#15014 (still shows removed imperihome)

Maybe I misunderstand the concept of next.openhab.org. My expectation would be that it is generated automatically every few hours based on the current main branch. Is there a manual interaction required to trigger the generation? Where can I find Jenkins the corresponding Jenkins job?

[kaikreuzer]

The reason is that https://ci.openhab.org/view/Documentation/job/gather-external-docs/ stopped working in April and neither @Confectrician nor myself were able to figure out the reason for this...

[holgerfriedrich]

Apparently wget on the test node is not working due to broken dependency on SSL library. Where can I find the script which is executed?

I not able to view the Jenkins job config. Could you share it please?

[kaikreuzer]

It is executing this script as a pre-execution step:

ln -s libssl.so.3 libssl.so
chmod +x update-external-resources.sh
./update-external-resources.sh

[holgerfriedrich]

Thanks @kaikreuzer. Unfortunately I cannot locate update-external-resources.sh. Is it part of one of the git repos?

From the log it is hard to deduce what is actually happening. Did you already try to enable debug logging in the shell script (set -xv)?

...and why do we need this hack using ld_library_path to run wget?

[kaikreuzer]

I just spent the whole evening on investigating this, only with little success... The script update-external-resources.sh is in the final branch. The script calls wget, which requires openssl for the HTTPS handshake and fails loading the according shared library. The root cause seems to be that host (with Ubuntu 22.04) has OpenSSL 3.0.2 installed, while within the Jenkins built-in node (which runs in a docker container jenkins/jenkins:2.387.3) shows OpenSSL 1.1.1. I unfortunately do not really know what to do from here...

[kaikreuzer]

...and why do we need this hack using ld_library_path to run wget?

We don't - you probably only saw this in one of the execution logs from tonight, where I tried different ways to figure out what's happening...

[holgerfriedrich]

Cannot help much with Docker.... But did you try if curl would be able to connect? It also uses openssl, so maybe it has the same problem...

[Confectrician]

I remember switching this script from curl to wget ages ago, because it caused build problems too. I also did further research in the last weeks but was not able to find a suitable solution yet.

I am thinking of doing some different approach. The only thing we need is one file, so maybe we could store the latest one needed in the final branch in another way. It is not needed to do this in the particular script.

@kaikreuzer maybe we could set up a webhook for this repository which runs a github action to get the file from jfrog artifactory. The current CI build is triggered from openHAB distribution, so we could add the configured webhook to the post build tasks over there.

This way we could omit that part completely in the update external ressources script.

WDYT?

[Confectrician]

From a quick search, something like https://github.com/suisei-cn/actions-download-file could work.

A github action can be calles with the workflow dispatch event via rest-api. https://docs.github.com/en/webhooks-and-events/webhooks/webhook-events-and-payloads#workflow_dispatch

[holgerfriedrich]

@Confectrician Another option could be to use a pipeline script in Jenkins. Http(s) requests can be done from those scripts.

Not sure what is less effort to implement. Especially if the OH CI does not use pipeline scripts yet.

[Confectrician]

To be honest i am not a jenkins pro. I can walk around and look at the logs and do simple twerks.

Did you work with pipeline scripts already? What would be the workflow and what would happen with the file in thise case? We need it in the repository during the build for processing all addon doc files.

Generally i am aiming for reducing the external build dependencies wherever possible, so fetching the file via github action may be the more solid approach, even it is some extra effort at the beginning.

[holgerfriedrich]

@Confectrician I had a look into Jenkins. The problem I see is that you want a recursive get over the folder structure. If we could name the target file, there would be two options: 1) (easy) Install "HTTP Request" plugin. Add an additional Build step (before) you call the script from Git, HTTP Request, configure extra option "Output response to file" and put a file name. After the build step, the file will be in the workspace. 2) Pipeline offers a bit more options. Keeping it simple: Create a new job (pipeline, you cannot use the existing freestyle job), put in a simple 2-stage pipeline: First stage to get the file, second stage to run the shell script.

I can assist, but can we get the correct filename without sweeping through the full folder structure? I don't know if there is a plugin for this.

[Confectrician]

I have no idea how we should know the filename directly.

For example https://openhab.jfrog.io/ui/native/libs-snapshot/org/openhab/distro/openhab-addons/4.0.0-SNAPSHOT/ currently has two file sets for two different builds. We always use the latest one.

We could limit it to the current snapshot folder, but this was exactly what i wanted to avoid, because we would have to adapt the folder on each openHAB release. With just downloading some files and choosing the latest one it doesn't matter which version is currently active on distro.

[holgerfriedrich]

Ok, then lets focus on how to get the wget (or curl) working properly in the container... can you share how the container is set up and run? (I cannot see the job config, maybe you could share via PM).

[Confectrician]

I will send you some information via openHAB community message.

[Confectrician]

First sorry for the long waiting period Apparently it seems, that the ci build is running fine again currently, without fixing anything.

Therefore i would ask you to check if the current website state is reflecting everything as expected from your side. I triggered a manual website build for next.openhab.org which should be public already. If everything is up to date again, i would vote for "never touch a arunning system" at least till august, when we have a new major version releases and may focus on optimizing this process in general.

[holgerfriedrich]

@Confectrician thanks, it seems everything I mentioned above is now resolved. Closing issue.