search

msys2 / setup-msys2

GitHub Action to setup MSYS2
https://github.com/marketplace/actions/setup-msys2
MIT License
304 stars 40 forks source link

integrate with the tool cache somehow? #145

Open jeremyd2019 opened 3 years ago

jeremyd2019 commented 3 years ago

Github Actions has an API for working with the tool cache: https://github.com/actions/toolkit/tree/main/packages/tool-cache

This seems like a reasonable fit for this action. I don't know if just the installer download should be cached, or if the install should live in the tool cache, as the tool cache is preserved between runs, unlike the runner _temp directory where this stuff goes now.

lazka commented 3 years ago

Why not.

Unclear re whole install. In theory you can put your msys2 install at C:/msys64 and pass "release: false" to the actions, so it uses the "default" msys install. This way it will reuse things already.

jeremyd2019 commented 3 years ago

Well, it's really meant for tools that are versioned, and keeping track of different versions of tools side-by-side. Like the setup-python action does.

lazka commented 3 years ago

Yeah, re unclear I mean that it would keep state between runs. Currently it doesn't and maybe someone depends on it (??)

jeremyd2019 commented 3 years ago

only in self-hosted runners. github-hosted runners are apparently always fresh and clean for every run

lazka commented 3 years ago

we can try. from what I see it copies everything into the cache, so we need to check if that slows down the hosted runners.

https://github.com/actions/toolkit/blob/45647689407e7fb224e06d066dde6aefa67a365f/packages/tool-cache/src/tool-cache.ts#L434

jeremyd2019 commented 3 years ago

Unclear re whole install. In theory you can put your msys2 install at C:/msys64 and pass "release: false" to the actions, so it uses the "default" msys install. This way it will reuse things already.

This seems like a better idea for the whole install. I need to look at my fork and make sure that that would work with C:/msys32. I don't think I handled that case because the hosted runners didn't have that present anyway.

jeremyd2019 commented 3 years ago

I'm just grasping at straws to reduce the approx 1.5 to 1.75 hours it takes to get to 'Process build queue' on my raspberry pi 3B+ self-hosted runner. I suspect a large part of it is the slowness of the SD card, but I was unable to get any usb storage to work with a quick test.

Biswa96 commented 3 years ago

Are you booting from SD in RPi? Have you tried this https://www.worproject.ml/downloads ?

jeremyd2019 commented 3 years ago

I am booting from SD. I don't really have a suitable USB storage device for it, and attempting to add a usb mass storage device to the running install resulted in errors, so I don't hold out much hope that it will work. Also, I thought that booting from USB was a pi4 enhancement.

I used https://github.com/WOA-Project/WOA-Deployer-Rpi

ZachBacon commented 3 years ago

I am booting from SD. I don't really have a suitable USB storage device for it, and attempting to add a usb mass storage device to the running install resulted in errors, so I don't hold out much hope that it will work. Also, I thought that booting from USB was a pi4 enhancement.

I used https://github.com/WOA-Project/WOA-Deployer-Rpi

I think it has to be supported not just from the raspberry Pi itself but by the bootloader used.

eine commented 3 years ago

@jeremyd2019 if I don't get it wrong, the feature your are requesting is implemented already, but disabled by default. See https://github.com/msys2/setup-msys2/commit/434d91ba7b60de97cd851824772741e4d55b9223, #61 and #50.

jeremyd2019 commented 3 years ago

Not quite. That cache is stored with github, so for a self-hosted runner still requires downloading (and uploading, which is failing for me for the packages cache). The tool cache is local to the runner and persists between runs (again, for a self-hosted runner).

eine commented 3 years ago

@jeremyd2019, I see. I was confused because we do have tool-cache as a dependency (https://github.com/msys2/setup-msys2/blob/master/package.json#L33), but we use it for downloading the tarball/installer only.

I think the solution would to treat this orthogonally:

Does it make sense to you?

jeremyd2019 commented 3 years ago

yes. The PR I opened (#147) just caches the installer. One thing that is kind of annoying with a rolling release like msys2 is that the tool cache expects a (semantic) version, I just used the date of the installer.

jeremyd2019 commented 3 years ago

Maybe the version for the install could be a (small) constant, just so there is something (since it's in the directory layout)? Like "1" or "a"? Just brainstorming

jeremyd2019 commented 3 years ago

Turns out that if the version doesn't match what semver wants (major.minor.patch all numeric, plus some other stuff), it fails to find the previously cached tool. It looks like a form of 0.0.1-2021-07-05 would be valid, but to be safe I am looking at 0.0.20210705. But for an install, the shortest valid semver would probably be 0.0.1

jeremyd2019 commented 3 years ago

I've switched to release: false and providing a C:\msys32 install on my self-hosted runner for now.

eine commented 3 years ago

FTR, https://github.com/msys2/setup-msys2/issues/145#issuecomment-874419391 was fixed in #148.