Add ability to use GitHub Actions to build NVDA

dpy013 commented 4 years ago

hi The appveyor currently used by NVDA cannot compile multithreaded when compiling NVDA

Is your feature request related to a problem? Please describe.

If possible, we can compile NVDA with GitHub Actions

Describe the solution you'd like

help docs GitHub Actions docs and Events that trigger workflows - GitHub

Describe alternatives you've considered

none

Additional context

Image	YAML Label	Included Software
Ubuntu 24.04 ^beta	`ubuntu-24.04`	[ubuntu-24.04]
Ubuntu 22.04	`ubuntu-latest` or `ubuntu-22.04`	[ubuntu-22.04]
Ubuntu 20.04	`ubuntu-20.04`	[ubuntu-20.04]
macOS 14	`macos-latest-large` or `macos-14-large`	[macOS-14]
macOS 14 Arm64	`macos-latest`, `macos-14`, `macos-latest-xlarge` or `macos-14-xlarge`	[macOS-14-arm64]
macOS 13	`macos-13` or `macos-13-large`	[macOS-13]
macOS 13 Arm64	`macos-13-xlarge`	[macOS-13-arm64]
macOS 12	`macos-12` or `macos-12-large`	[macOS-12]
Windows Server 2022	`windows-latest` or `windows-2022`	[windows-2022]
Windows Server 2019	`windows-2019`	[windows-2019]

thanks

dpy013 commented 4 years ago

cc @josephsl and @michaelDCurran system config （E5 2vCPU/7G RAM）

LeonarddeR commented 4 years ago

I have investigated github actions for add-on compilation, and it wasn't suitable at that time because of https://github.com/actions/upload-artifact/issues/3. In short, Github actions uploads artifacts in a zip file. Even when that would be supported, wildcards aren't either, see https://github.com/actions/upload-artifact/issues/11.

Having said that, it will probably take hours or days to convert our appveyor script to Github Actions, thereby not gaining much benefit. Closing as can't fix for now. We can always reopen if desired.

dpy013 commented 4 years ago

I have investigated github actions for add-on compilation, and it wasn't suitable at that time because of actions/upload-artifact#3. In short, Github actions uploads artifacts in a zip file. Even when that would be supported, wildcards aren't either, see actions/upload-artifact#11.

Having said that, it will probably take hours or days to convert our appveyor script to Github Actions, thereby not gaining much benefit. Closing as can't fix for now. We can always reopen if desired. Thank you for your instructions

XLTechie commented 4 years ago

One of my back burner projects has been to convert the Appveyor script to something more generic. The goal being so that people who fork the main repo, do not actually need a source compilation environment to test their own PRs and private changes, or otherwise build from source for their own purposes.

I haven't gotten far with it because of lack of time, but it seems to me that if we could get this working as an Appveyor alternative, it would make that idea ultimately a bit more straight forward.

Edit: implemented in #16221, #16293.

LeonarddeR commented 4 years ago

I agree that appveyor is not very ideal, however, our current appveyor script uses pretty specific functionality, also related to tests. It will take some time before actions is mature enough.

dpy013 commented 3 years ago

cc @seanbudd

dpy013 commented 3 years ago

I have investigated github actions for add-on compilation, and it wasn't suitable at that time because of actions/upload-artifact#3. In short, Github actions uploads artifacts in a zip file. Even when that would be supported, wildcards aren't either, see actions/upload-artifact#11.

Having said that, it will probably take hours or days to convert our appveyor script to Github Actions, thereby not gaining much benefit. Closing as can't fix for now. We can always reopen if desired.

Has the above problem been solved by github actions?

dpy013 commented 2 years ago

appveyor has started to limit the number of downloads to. {"message":"Artifacts download limit (1024 MB/day) exceeded."} test method is as follows: first open this address Once opened, the above prompt will appear {"message":"Artifacts download limit (1024 MB/day) exceeded."}

LeonarddeR commented 2 years ago

See #13281

LeonarddeR commented 2 years ago

As actions is much more mature now and a switch to actions could be a possible way to avoid #13281, I"ll reopen this until a solution for #13281 has been chosen.

LeonarddeR commented 2 years ago

never mind, this is still a showstopper for us:

During a workflow run, files are uploaded and downloaded individually using the upload-artifact and download-artifact actions. However, when a workflow run finishes and an artifact is downloaded from either the UI or through the download api, a zip is dynamically created with all the file contents that were uploaded. There is currently no way to download artifacts after a workflow run finishes in a format other than a zip or to download artifact contents individually.

dpy013 commented 2 years ago

I have investigated github actions for add-on compilation, and it wasn't suitable at that time because of actions/upload-artifact#3. In short, Github actions uploads artifacts in a zip file. Even when that would be supported, wildcards aren't either, see actions/upload-artifact#11.

Having said that, it will probably take hours or days to convert our appveyor script to Github Actions, thereby not gaining much benefit. Closing as can't fix for now. We can always reopen if desired. hi Maybe Creepios/sftp-action: GitHub Action to upload files/directories via ssh This problem can be solved

XLTechie commented 6 months ago

@LeonarddeR Your show-stopper seems no longer to be the case. Can you confirm that your concerns are satisfied?

LeonarddeR commented 6 months ago

After https://github.com/nvaccess/nvda/issues/13281, snapshots are no longer directly downloaded from appveyor. This means that the fact that github only offers zip files is less relevant, though it is still a bit of a complicating factor when downloading artifacts from pull requests. That said, I think there's enough potential to reopen this.

seanbudd commented 5 months ago

Hi, just to give an update from NV Access. Moving to GitHub actions is on our roadmap due to many issues with AppVeyor:

AppVeyor instances appear to be getting slower, slowing the build times
We can only perform one build at a time
GitHub actions allows us to split up the CI/CD pipeline (e.g. the lint check can run independently of the build)
GitHub actions are easier for forks to use
being in one platform (i.e. GitHub) is generally simpler

XLTechie commented 5 months ago

@seanbudd wrote:

AppVeyor instances appear to be getting slower, slowing the build times

We have long had the ability to make those significantly faster, by caching already compiled components and deploying them to the run environment through AppVeyor's caching options.

I often wondered why NV Access didn't, especially for the slow-to-compile C components, and clonable items that hadn't been updated, but assumed there was a reason.

Of course, Actions can do that, too.

dpy013 commented 5 months ago

As a side note, GitHub actions can be used to compile and test NVDA code using relatively new windows systems. It would also be possible to link crowdin to the GitHub code repository. Finally, it might be possible to automate the task of posting alpha, stable, and test boards to the website. To reduce the workload of the staff!

seanbudd commented 5 months ago

@XLTechie

I often wondered why NV Access didn't, especially for the slow-to-compile C components, and clonable items that hadn't been updated, but assumed there was a reason.

We've already done this in a different way for some components like https://github.com/nvaccess/nvda-cldr

The issue of AppVeyor making their instances cheaper and slower over time is still going to be an issue even if we optimize other parts of the build system. The slowest parts of the build are quite challenging to split up. We can always speed up the build system in many ways, and GitHub actions is probably the most effective solution.

XLTechie commented 5 months ago

@dpy013 You are right about the relatively new Windows systems. It might even allow for broader testing setups, if we can cause artifacts to be tested on several versions of Windows at once. That alone would probably make it worth doing.

Certainly, if Crowdin can be linked to AppVeyor compiles as it is now, it can be equally well linked to Actions compilations, probably with the same scripts.

Posting artifacts to the website is already automated, at least for alphas and betas.

nvdaes commented 5 months ago

I've started to study modules of GitHub Actions with the Study guide for a possible preparation to get the GitHub Actions certificate from GitHub. I don't know if finally I'll try to get the certifications since I need to show Government documentation, installing a software etc. But if someone doesn't take this, I may create a PRafter testing. We may use the new attestations feature too.

seanbudd commented 5 months ago

@nvdaes - there's no need to worry about build signing, we're currently working on changing this actively

nvaccess / nvda