search

msys2 / msys2-installer

The one-click installer for MSYS2
BSD 3-Clause "New" or "Revised" License
590 stars 88 forks source link

MSYS2 scripted Install, GHA, etc #5

Closed MSP-Greg closed 2 years ago

MSP-Greg commented 4 years ago

I'm not sure where to post this. The issue revolves around scripting MSYS2 installation and update.

Some other recent issues (https://github.com/msys2/MSYS2-packages/issues/1962, https://github.com/msys2/MSYS2-packages/issues/1965) have touched on the issue.

I assume most of the breakage will be caused by commits to https://github.com/msys2/MSYS2-packages.

GitHub Actions (GHA) now has MSYS2 installed on their Windows images. Both 64 & 32 bit clang cmake llvm toolchain ragel packages/groups are installed. Custom Actions have also been developed to install/update MSYS2. Of concern is that the script GHA is using will reliably update MSYS2. Currently, the images are updated frequently, typically every 10 days or better.

Also mentioned is moving some CI to GHA. I think there are several people using MSYS2 that would be willing to help. GHA supports jobs that conditionally run based on file locations, so specific jobs could be ran if, for instance, gpg, msys2-runtime or pacman files were changed.

I have mostly worked with MinGW build scripts (and not a lot), so I'm not familiar with the MSYS2 code, but I'd certainly be willing to help with GHA workflows if they will help with CI.

Also, GHA isn't the only CI, so a 'recommended' script would be helpful.

This is a very general issue, but making MSYS2 robust as far as scripted install/update is important.

MSP-Greg commented 4 years ago

Thinking back on this, the last time I recall issues with MSYS2, and the issues listed above -

Package A and package B get updated. Doing a normal update (via -Syu) installs A first, then B. But, due to whatever, B must be installed first, then A, as A requires the new B?

Not sure if that describes one possible type of problem, but if it does, that might be messy to check with CI?

lazka commented 4 years ago

So, we need

eine commented 4 years ago

@lazka, I've been enhancing and maintaining eine/setup-msys2 for the last months. It's a GitHub Action for users to easily setup MSYS2 in their workflows.

Since GitHub added a default MSYS2 installation to windows-latest virtual environments some days ago, Action setup-msys2 allows to either reuse that, or to retrieve and extract a clean one from https://github.com/msys2/msys2-installer/releases.

The repository includes a workflow to test multiple (+15) setups: https://github.com/eine/setup-msys2/blob/master/.github/workflows/action.yml. For examples of other external projects using it (ref eine/setup-msys2#20):

Hence, I'd like to propose forking setup-msys2 or moving it to this organization:

lazka commented 4 years ago

Hence, I'd like to propose forking setup-msys2 or moving it to this organization:

Sounds good to me (without knowing the differences between the repos). @Ecco any thoughts on this?

Ecco commented 4 years ago

Hi @lazka ! We'd be glad to transfer ownership of our numworks/setup-msys2 repo to you guys 😄

lazka commented 4 years ago

Hi @lazka ! We'd be glad to transfer ownership of our numworks/setup-msys2 repo to you guys 😄

Any thoughts on using @eine 's fork instead? -> https://github.com/eine/setup-msys2/blob/master/CHANGELOG.md

eine commented 4 years ago

@lazka, as a matter of fact, >65% of changes and 50% of the actual code in numworks/setup-msys2 were contributed from eine/setup-msys2 through PRs. The reason for eine/setup-msys2 to be 26 commits ahead of numworks' is that the latter has been kind of stalled for the last 5 months.

Anyway, I'm ok with having numworks/setup-msys2 transferred and then proposing PRs to bring it up to date. Specially, because numworks/setup-msys2 contains more issues/PRs.

eine commented 4 years ago

@lazka, I think that the "automatic cleanup" of intermediate updates is not working as expected (ref https://github.com/msys2/MSYS2-packages/issues/1962#issuecomment-632120904). As you can see in https://github.com/eine/setup-msys2/runs/743849758?check_suite_focus=true#step:4:182, :: To complete this update all MSYS2 processes including this terminal will be closed. Confirm to proceed [Y/n] warning: terminating MSYS2 processes failed is shown and the return code is !=0. I worked around it with https://github.com/eine/setup-msys2/commit/258a810be1a74f18a69ec0ce15455da0198de8a0, but I thought that you'd want to be aware.

EDIT

Maybe it should be already fixed by msys2/MSYS2-packages#2002?

lazka commented 4 years ago

Maybe it should be already fixed by msys2/MSYS2-packages#2002?

could be, but unclear. I'll watch out if I see it after this.

eine commented 4 years ago

@lazka, @Ecco, shall we migrate numworks/setup-msys2 or eine/setup-msys2?

lazka commented 4 years ago

@eine feel free to transfer eine/setup-msys2 to the msys2 org. I'll make you both admins there in case that isn't already the case then.

eine commented 4 years ago

@lazka, I'm afraid I cannot, because I'm not allowed to create repos in this org. I invited you to the repo, so you can hopefully trigger the transfer.

lazka commented 4 years ago

hm, right, I'm not allowed either. Can I get admin rights maybe?

eine commented 4 years ago

It seems not possible to give admin rights in "user repos". All I could do was add you as a collaborator, which seems not enough: https://help.github.com/en/github/setting-up-and-managing-your-github-user-account/permission-levels-for-a-user-account-repository

So, I "requested" the transfer to your personal account. Hopefully, once it's lazka/setup-msys2, you will be allowed to transfer it to the org.

lazka commented 4 years ago

thanks, done

eine commented 4 years ago

Nice! I updated the readme to replace all the references from 'eine/setup-msys2' to 'msys2/setup-msys2' (README.md).

lazka commented 4 years ago

I've contacted github to remove the "forked from" status (and opened a few PRs ;) )

eine commented 4 years ago

Oh, I was wondering why it was gone :D. All merged now. Thanks.

lazka commented 4 years ago

So the only thing left imo is that we document what we do in msys2/setup-msys2 on the website.

eine commented 4 years ago

Should it be a section in https://github.com/msys2/msys2.github.io/blob/source/docs/wiki/MSYS2-installation.md?

lazka commented 4 years ago

I was more thinking about a dedicated "CI" section with

lazka commented 4 years ago

Here is a first draft: https://www.msys2.org/docs/ci/

lazka commented 4 years ago

Is there anything left to do here? I'd suggest to open separate actionable issues if there is.

eine commented 4 years ago

According to the OP's comment above:

Also, GHA isn't the only CI, so a 'recommended' script would be helpful.

This is a very general issue, but making MSYS2 robust as far as scripted install/update is important.

Most of the current setup-msys2 Action can be rewritten from JavaScript to Powershell. That would allow to use the same script for the GitHub virtual-environment, the GitHub Action, or for users to run it on other services. @MSP-Greg are you up to integrating https://github.com/actions/virtual-environments/blob/master/images/win/scripts/Installers/Install-Msys2.ps1 into setup-msys2?

MSP-Greg commented 4 years ago

@eine

So the goal is a PowerShell script?

Re https://www.msys2.org/docs/ci/, when MSYS2 is pre-installed, I tend to prefer updating only what is needed, as opposed to a full update. Especially with Action, which is updating frequently, unlike other CI services that often went months between updates...

eine commented 4 years ago

The main entrypoint of a GitHub Action must be a JavaScript script. However, it is possible to use it for executing a powershell script only; i.e. as an intermediate argument parsing layer only. For example, currently we use powershell to compute the checksum of the installer: https://github.com/msys2/setup-msys2/blob/master/main.js#L51

Hence, yes, the goal would be to migrate part of the logic that is currently written in JS to powershell, so it is usable outside of GitHub Actions.

Re msys2.org/docs/ci, when MSYS2 is pre-installed, I tend to prefer updating only what is needed, as opposed to a full update. Especially with Action, which is updating frequently, unlike other CI services that often went months between updates...

With setup-msys2, updating any package (including the base set of packages) is optional. Any proposed script should be feature-equivalent to the current setup-msys2.

lazka commented 4 years ago

Re msys2.org/docs/ci, when MSYS2 is pre-installed, I tend to prefer updating only what is needed, as opposed to a full update.

Partial updates are not supported, same as with Arch Linux.

lazka commented 2 years ago

I think there is nothing left to do here. I I missed something maybe copy and create a new issue.

MSP-Greg commented 2 years ago

@lazka

I apologize for leaving this open. Thanks.

lazka commented 2 years ago

I don't mind open issues :)