Public CI metaissue - Githubissues

dfinity / motoko

Simple high-level language for writing Internet Computer canisters

Apache License 2.0

517 stars 97 forks source link

Public CI metaissue #2565

Open nomeata opened 3 years ago

nomeata commented 3 years ago

This issue tracks various refinements we want to do to our CI infrastructure post-open source.

[x] https://github.com/dfinity/motoko/issues/2544 Pick a public CI system (Github actions + cachix for now)
[x] https://github.com/dfinity/motoko/pull/1224 Set it up
[x] https://github.com/dfinity/motoko/issues/2566 Automatically create releases and publish build artifacts there
[x] https://github.com/dfinity/motoko/issues/2567 Allow public access to drun, so that all tests can be run on CI This blocks most of the following items:
[ ] https://github.com/dfinity/motoko/issues/2568 Publish reports (profiling, documentation preview) from GitHub Action runs
[x] #2740 Report performance changes in comment to PR (maybe using this Github Action)
[x] #2744 Switch branch protection to Github Action rather than hydra
[ ] #2771 Stop using the internal CI system.

nomeata commented 3 years ago

Useful article about pull_request and pull_request_target, and the security implications of PRs from other repositories: https://securitylab.github.com/research/github-actions-preventing-pwn-requests/

nomeata commented 3 years ago

Discussing nix-based CI options on the Nix Discourse: https://discourse.nixos.org/t/github-actions-and-or-with-hercules-ci/13537

osa1 commented 3 years ago

"CI: Publish reports" this is not something we do today, right?

I'm thinking what's left to do before stopping using Hydra. I think once we port posting perf and size changes to GH CI we should be able ditch it?

nomeata commented 3 years ago

"CI: Publish reports" this is not something we do today, right?

Yes, we have direct links to the latest flame graph, documentation, etc. for each PR. Would be a shame to lose that, but maybe ok temporarily until we have a good idea. I should research what kind of services are there out there.

Besides that it's just the perf changes comments, which I can look into this week maybe

osa1 commented 3 years ago

Yes, we have direct links to the latest flame graph, documentation, etc. for each PR.

Interesting, I didn't know that. Where can I see these reports?

nomeata commented 3 years ago

You didn't know? Anyways, see https://github.com/dfinity/motoko/pull/2562 for some of the links, which we removed from the README to keep up appearances.

nomeata commented 3 years ago

Flame graphs at https://hydra.dfinity.systems/job/dfinity-ci-build/motoko/tests.profiling-graphs/latest/download/1/index.html I assume (I can't check, of course). One can construct similar urls for each PR. I hope to preserve that feature, or improve it (easier discoverability in particular of course). But needs not block turning off hydra.

nomeata commented 3 years ago

Maybe we can sign up to coveralls.io (free for open repos) and upload coverage data there. Supports OCaml (https://github.com/aantron/bisect_ppx#Coveralls). We used to have coverage reports, but they have somewhat bitrotted.

nomeata commented 2 years ago

We might very easily be able to speed up our CI system by hooking up to (and paying for) https://nixbuild.net/, with no noticable changes to our existing workflows.

~Would it be possible to get signed up for this? I'll then set it up.~

Oh, no Mac support it seems. Maybe not then.