Use 4.x-dev branch to reset the repo

[sgiehl]

Currently the builds are containing broken versions of the plugins that were newly moved to submodules. The reason is, that checking out the very old master branch kind of fails for those, causing all files that were included in the repo before to be removed when switching back to the release...

Using the 4.x-dev branch should fix that. I've also added a git clean -fd to ensure all files are removed that shouldn't exist. That might help when trying to do another release for 3.x, as the submodule problem might happen in the other direction then. Or are there any files that aren't on git, but should be present at that point in the script?

fixes https://github.com/matomo-org/matomo/issues/16517

[tsteur]

@sgiehl since the git clean -fd was never needed before I wonder if it could be removed? Just to make sure we don't regress anything.

f I understand things correctly would it be a problem to initially delete the cloned git to ensure we're always starting from scratch and there won't be an issue with submodules that already as a directory etc?

[sgiehl]

@tsteur Removed the git clean for now. When releasing 4.x versions that won't be a problem at all. But with the changed script it won't be possible to release new 3.x versions at all. I think it will fail when trying to switch from 4.x-dev to a 3.x tag. Guess we would need to change that completely, as even a fresh clone won't work then.

Guess we would need to checkout the tag with a force and do a git clean afterwards to remove the files that might remain...

[tsteur]

@sgiehl I was hoping that git clone --single-branch --branch <tag> would help here:

and then we do a rm -rf $LOCAL_REPO every time? That might help? Haven't tested it though. We still need to be able to safely build zip for different major versions

[sgiehl]

Yeah. Guess directly cloning the correct tag should actually do the trick and make the most following commands obsolete.

[mattab]

Btw The reason we don't clone each time is that it takes a very long time..

On Mon, 12 Oct 2020, 8:31 pm Stefan Giehl, notifications@github.com wrote:

Yeah. Guess directly cloning the correct tag should actually do the trick and make the most following commands obsolete.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/matomo-org/matomo-package/pull/118#issuecomment-706933793, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADR6TJOPHPATKK3W32IKSDSKKWEHANCNFSM4SKETWIQ .

[sgiehl]

@mattab using the --single-branch option limits all downloads to the specified branch only and should speed up the cloning

[tsteur]

@mattab also it's not like it's being executed so often and could just run in the background. Seems this is safest way to ensure correct data will be in release.

[sgiehl]

@tsteur changed the script so it always does a git clone...

[tsteur]

Ran parts of the scripts and test various things like existing tag vs not existing tag etc and looks good to me. Couldn't do a full test though as the build script doesn't run on Mac. Not sure how to best test things still work @sgiehl ? Might be good to set the "build only" flag (so it only generates a zip) and do a test install from this zip and check the install works well plus the file integrity check looks good?

[mattab]

Here is an idea to help us test this script, maybe it'd be good use of our time to directly also automate the process via a Github action, and test the code as a github action? https://github.com/matomo-org/matomo-package/issues/119 then we'd have automated builds forever, which would be amazing :rocket: (we'd just have to solve the issue of signing builds with a different key which shouldn't be too hard but will take a few hours)

[tsteur]

@mattab unfortunately automating it doesn't really help us test the script itself that much plus it's not a priority at the moment as we're working towards the Matomo 4 release.

[sgiehl]

I've run the script locally in my VM with the build-only option to test it. The archive seems to contain all relevant files and also the missing submodule contents. Building the archives took less than 2 minutes on my vm (including the git clone and without any composer cache). Manifest file seems to contain everything... Also tried building a 3.14.1 release, which also worked fine. (after disabling the check if the version already exists on the build server). So imho I guess the changes should be fine and should be used to build a new release that contains the missing submodule files...