search

FreeCAD / FreeCAD-Bundle

Stand-alone repo to Build and Deploy installable FreeCAD images. Do not open general FreeCAD issues here.
https://freecad.org
GNU Lesser General Public License v2.1
312 stars 63 forks source link

[Problem] Weekly builds "freecad_version.txt" content does not match version number in bundle #304

Open Murmele opened 3 weeks ago

Murmele commented 3 weeks ago

Describe the bug The version in the "freecad_version.txt" is quite actual. But the version number of the bundles is much lower and when installing the version it can be also seen the number of the bundle.

grafik

Content of the "freecad_version.txt":

rev_number: 38693 (Git)
branch_name: main
commit_date: 2024/09/10 18:06:05
commit_hash: 85ade21436d4ce618691720688b299c9b6902590
remote_url: https://github.com/FreeCAD/FreeCAD main
OS: Windows 11 build 22621
Word size of FreeCAD: 64-bit
Version: 1.0.0RC1.38643 (Git)
Build type: Release
Branch: main
Hash: fe40748c10503eed367840239db3e0f81873e359
Python 3.11.9, Qt 5.15.13, Coin 4.0.2, Vtk 9.2.6, OCC 7.7.2
Locale: English/United States (en_US)
oursland commented 3 weeks ago

The file freecad_version.txt refers to the version of the freecad_source.tar.gz file. I do understand the confusion; I had to look up how these files were created to figure it out.

Murmele commented 3 weeks ago

Is it possible to create bundles from the version in the freecad_version.txt instead of just a weekly? A nightly would be already nice, because my PR got merged a few days ago but now I have to wait for a complete week to get a bundle to give it a friend.

oursland commented 3 weeks ago

Currently the Bundle is built once a week. I agree that nightly builds would be preferable, but I am unfamiliar with the costs of doing so.

The source is generated every 6 hours, or 4 times a day.

@adrianinsaval A couple questions:

  1. Would nightly builds be possible, and if so, are there any costs to doing so?
  2. Why is the source generated out-of-sync with the built artifacts?
adrianinsaval commented 3 weeks ago

1 - there is not a monetary cost to us, we use github's free runners. The builds are compute intensive though because they are clean builds with no caching so they take up to 2.5h. Doing this daily for little benefits seems a bit abusive IMO.

In any case while we call them weeklies because we kept the original name, nowadays they happen twice a week actually on monday and friday nights, this was deemed a good compromise in the past. It is worth nothing that github seems to limit how many ci runs you run concurrently. I noticed that even the source generation workflow took a while the other day because main's ci wasn't using caching properly so there where many long workflows running at the same time. Making these happen daily might slow down other more important workflows. One more thing to consider is that storage space in the conda repo is limited and currently old builds are not cleaned automatically, If builds where to happen daily this will fill up much more often than now so this should be automated.

So I have to ask, is there real benefit in having daily builds as opposed to twice a week? I'm not opposed to the idea though.

2 - loooo's idea IIRC was that these source files could be used by other packagers that would like to do dev releases and wanted an up to date source package without using git, with proper version info and nowadays they also include submodules. The source package creation workflow is fairly short so it's not much of an issue to run it multiple times, this workflow also pushes the source with version info and submodules to a dummy repo in salsa.debian.org that is used for the ubuntu ppa daily builds.

adrianinsaval commented 3 weeks ago

as for the OP, this is expected. I'm open to suggestions about how to document this to avoid confusion but due to the order in which these files are created even if we did match their schedules it might be the case that the version won't match, for example when a build fails. Version info is displayed in the filename already so users should not pay attention to freecad_version.txt.

We could also just remove freecad_version.txt as I think it's not really required for anything in particular.

adrianinsaval commented 3 weeks ago

ah one more thing, the arm bundles are not currently done by github runners but by cirrus-ci https://cirrus-ci.com/github/FreeCAD/FreeCAD-Bundle which is also free but I believe does have a limit on how many minutes per month you can use, so that probably shouldn't be moved to a daily schedule. I think it should be doable to port this back to github though, by using qemu for the little portion that needs to run an arm binary or eliminating the need to run the binary at all. Also github is supposedly going to eventually support linux arm too.

oursland commented 3 weeks ago

Thanks, @adrianinsaval ! That does clear up a lot of things for me.

I've been informed that some developers do not built locally, which is really bizarre to me. Instead they rely upon committing code, getting through code review, and waiting for the weekly build. I find this kinda wild, but I can see utility in having frequent builds to have access to bugfixes and other user-impacting functionality available to those users and devs who do not build locally.

The limitation on concurrent runners is a serious concern, so I'm not going to suggest increasing the frequency of builds until that limitation is removed.

adrianinsaval commented 3 weeks ago

I don't know what the actual limit is, there certainly can be several jobs at the same time. But I have noticed a few times that when there are too many running jobs new jobs take long to be assigned a runner

adrianinsaval commented 3 weeks ago

I've been informed that some developers do not built locally, which is really bizarre to me.

Indeed quite bizarre, one possible way to help with that could be to upload a compiled package in the ci workflow as an artifact with a conda environment file that would make it easy to reproduce the environment locally. That way devs that don't compile could download the artifact after pushing a commit and then install and test locally, that would also mean that a PR would get tested before merge

oursland commented 3 weeks ago

Someone has requested this in the past for PRs. I was under the impression that there's space limitations for the GitHub storage. What would the lifespan of such an artifact be?

Murmele commented 3 weeks ago

I've been informed that some developers do not built locally, which is really bizarre to me.

Indeed quite bizarre, one possible way to help with that could be to upload a compiled package in the ci workflow as an artifact with a conda environment file that would make it easy to reproduce the environment locally. That way devs that don't compile could download the artifact after pushing a commit and then install and test locally, that would also mean that a PR would get tested before merge

Yes this would be also quite nice, so everybody can test a pullrequest before it gets merged, without having to compile the project by him self.

luzpaz commented 3 weeks ago

So I have to ask, is there real benefit in having daily builds as opposed to twice a week? I'm not opposed to the idea though.

I think during the RC process we should consider doubling that (4 times a week). Reasons:

Edit: Case in point: https://github.com/FreeCAD/FreeCAD/issues/16479#issuecomment-2346415565