iTowns bundle is sometimes not found in tests workflow

[jailln]

I recently sometimes experience the following error during the CI:

https://github.com/iTowns/itowns/actions/runs/6000288882/job/16510986670?pr=2172

It does not appear to be deterministic so a workaround is to run the tests again.

@mgermerie Do you think it can be related to the recent changes on the test workflow ? (#2166)

[mgermerie]

Yes this is related to what I did in #2166.

Github action allows to store some data in what is called an artifact. This can be used to share data between different jobs. For iTowns, we upload to github action server the bundle created in Build bundle, check Linter and generate documentation job. It is later downloaded in Deploy to itowns.github.io and Rekease GitHub jobs.

Since #2166, functional tests are no longer in the same job as the build. Therefore we need to download the artifact uploaded in build job in functional tests jobs as well.

To prevent having too many unused artifact clogging GitHub action server, we also have a job to frequently remove artifacts from server (the script is here).

Therefore, if you restart functional tests a while after the latest build job, the build artifact could have been deleted. To solve this, it is necessary to relaunch all jobs (and not only functional tests).

Maybe we could update remove-old-artifact job to change the frequency between two artifacts clearing ?

[mgermerie]

For information, you can see if an artifact is still available for a workflow. It appears in the bottom of its summary webpage.

[jailln]

If it is recreated each time at the Build bundle, check Linter and generate documentation stage, it should always be avaible to the other downstream steps of the integration workflow?

[mgermerie]

Provided the build job ran not too long ago, the artifact should indeed always be available to the other jobs. The issue only occurs if remove-old-artifacts workflow is launched between the latest build job and the unit/functional tests job.

[jailln]

From my understanding, the build job is run for each workflow so every time functional tests are run, the build job has run juste before. Therefore, it should always be available to the functional tests except if the remove-old-artifacts job runs between the build step and the download step of a workflow which seems unlikely. Do I miss something?

[mgermerie]

From my understanding, the build job is run for each workflow so every time functional tests are run, the build job has run juste before. Therefore, it should always be available to the functional tests except if the remove-old-artifacts job runs between the build step and the download step of a workflow which seems unlikely. Do I miss something?

Yes you're right. I would just temper the unlikelyness : it can happen each time functional tests are manually and individually relaunched a while after the latest build job ran. But this should not be an issue in my opinion since the person who wishes to individually relaunch functional tests can also relaunch build of needed (and actually relaunch all jobs).

[jailln]

From my understanding, the build job is run for each workflow so every time functional tests are run, the build job has run juste before. Therefore, it should always be available to the functional tests except if the remove-old-artifacts job runs between the build step and the download step of a workflow which seems unlikely. Do I miss something?

Yes you're right. I would just temper the unlikelyness : it can happen each time functional tests are manually and individually relaunched a while after the latest build job ran. But this should not be an issue in my opinion since the person who wishes to individually relaunch functional tests can also relaunch build of needed (and actually relaunch all jobs).

Aaah yes ok I understand now :) Then I think we should do what you suggested: "Maybe we could update remove-old-artifact job to change the frequency between two artifacts clearing ?"

[mgermerie]

Some documentation on our integration processes would be useful as we already discussed :) I should tend to do it.

Thinking about it I wonder if it is really relevant to call remove-old-artifacts job more often. This induces more data stored on GitHub Action server. There are a few fact to consider :

• remove-old-artifacts job currently runs each day at midnight.
• We could rely on the developer, whose PR associated workflow did not run the day he/she wishes to run only functional tests job. He/she can see in the checks tab of his/her PR that there is no artifact uploaded - since it was deleted at midnight. Therefore, the developer only has to click the Re-run all jobs button instead of the clicking re-run button only for functional tests. This point however implies a more complete documentation on our integration script.
• The amount of data we store with our artifacts on GitHub action is relatively small. You can see here the list of artifacts stored. In example, at the time of this comment, it represents 7 artifacts weighing about 6.2 MB each.

With these points, should we reduce the frequency of remove-old-artifacts calls and if so, what frequency do you think we should use ?

[jailln]

Ok thanks for the precisions, my opinion is that we should leave it like it is for now, and rely on the developer to either know it or find this issue which will act as documentation until you write it. Therefore we could close this issue for now since it is not critical and there is a workaround. If it becomes more problematic at some point we can still get back at it.