search

pest-parser / pest

The Elegant Parser
https://pest.rs
Apache License 2.0
4.62k stars 259 forks source link

Migrate off of Travis-CI.org #482

Closed CAD97 closed 3 years ago

CAD97 commented 3 years ago

https://docs.travis-ci.com/user/migrate/open-source-repository-migration#q-when-will-the-migration-from-travis-ciorg-to-travis-cicom-be-completed

Q. When will the migration from travis-ci.org to travis-ci.com be completed? #

A. In an effort to ensure that all of our users - whether you build open-source, public or private repositories - receive regular feature updates, security patches and UX/UI enhancements, we are announcing that travis-ci.org will be officially closed down completely no later than December 31st, 2020, allowing us to focus all our efforts on bringing new features and fixes to travis-ci.com and all of our awesome users like yourself on the travis-ci.com domain.

Q. What will happen to travis-ci.org after December 31st, 2020? #

A. Travis-ci.org will be switched to a read-only platform, allowing you to see your jobs build history from all repositories previously connected to travis-ci.org.

The .org version of TravisCI will be retired by the end of the year. We can migrate to the .com portal or take the opportunity to migrate to Github Actions instead.

I believe @dragostis needs to hit a button in the Travis portal to migrate to the .com servers. I'd be willing to review/merge a migration to GHA, though I don't have the time currently to actually write the required configs to do the migration myself.

CAD97 commented 3 years ago

Also, it seems that the feedback from CI to bors is broken as well, so that needs to get fixed.

CAD97 commented 3 years ago

More: https://blog.travis-ci.com/2020-11-02-travis-ci-new-billing

So the free OSS tier may be going away on Travis? Then we definitely should migrate to GHA.

MaxBondABE commented 3 years ago

Hi,

I'd like to help out this this issue. My current plan is to migrate the .travis.yml file to equivalent GHA workflows using actions-rs. I will complete the task this weekend.

Please let me know if any additional considerations have cropped up since this issue was last updated.

Thanks, Max

MaxBondABE commented 3 years ago

I've implemented the logic in the above PR. I've left it as a draft, because I have several questions.

I have several questions:

Additionally, I have several notes:

Please let me know if you'd like me to change this PR.

Thanks, Max

MaxBondABE commented 3 years ago

It may be of interest to see a run of the CI in my fork.

https://github.com/MaxBondABE/pest/runs/1467325132?check_suite_focus=true

I did not execute the fuzzing logic, as I don't have a Fuzzit account.

CAD97 commented 3 years ago

@MaxBondABE

The Travis logic used the trying and staging branches. Is this still desirable? I glanced at the top few PRs, and they seem to target master.

We're using bors, so PRs target at master, but PRs are merged via bors, which pushes to staging to test before pushing to master.

The Travis logic pins versions of staging and nightly. Is this really necessary/desirable? It seems like it'd require more frequent maintenance.

When denying warnings, it's generally desirable to pin the toolchain version, so new warnings don't break the CI. This also serves as a (soft) reminder to (not) bump the minimum required toolchain version with new releases.

In the Travis version, after minimizing dependency versions, the code is retested in nightly. It is initially tested in stable. Am I mistaken, or should it be retested in stable instead of nightly?

The main reason for this is that nightly is required to generate the lock file with minimal versions. By testing on that nightly, we can avoid having to install two toolchain versions on that builder.

The job takes 10 minutes, half of which is spent installing Tarpaulin. This could probably be optimized further.

It looks like you're currently running all of the jobs sequentially. IIRC GHA supports 3 concurrent jobs for free OSS users, so adding some concurrency (even if it's just to run tarpaulin in its own job) would help speed up the build.

MaxBondABE commented 3 years ago

@CAD97

I've added the version pinning & parallelized the CI. It is ~40% faster (before adding the fuzz tests). I've also tested the Fuzzit integration & gotten it working.

Here is a run in my fork:

https://github.com/MaxBondABE/pest/runs/1468907355?check_suite_focus=true

The Fuzzit script was compiling scripts in release & attempting to execute them in debug. I've modified it so that it executes them in release. But I could modify if to go the other way if that's preferable.

I've realized that bors also needs to be reconfigured to use GHA. I'd be happy to write a workflow for bors as well, but it probably won't be until next weekend.

I'm moving the PR out of draft. Please let me know if there are any other changes you'd like to see.

Thanks, Max

MaxBondABE commented 3 years ago

While messing around with GitHub's tools, I discovered @SkiFire13 has done some work on this issue. They utilize caching in their workflow, and have migrated the bors configuration. Their workflow, sans fuzzing, is about 3x faster than mine.

https://github.com/SkiFire13/pest/blob/github-actions/.github/workflows/ci.yml

https://github.com/SkiFire13/pest/blob/github-actions/.bors.toml

SkiFire13 commented 3 years ago

Wow, looks like someone found my old attempt. That was also my first attempt at writing a CI configuration file, so take it with a grain of salt. In particular I'm not sure the bors configuration is right since I couldn't test it. I also didn't write jobs for fuzzing and coverage because I got a bit confused with env variables. This might also be why it is 3x faster.

MaxBondABE commented 3 years ago

Hi,

I apologize it's taken me so long to get back to this.

It seems that Fuzzit has been acquired by Gitlab. I have been unable to log into the platform (when I go to fuzzit.dev it redirects to https://about.gitlab.com/solutions/dev-sec-ops/ , and when I log in to Gitlab I'm unable to access Fuzzit).

I am uncertain that they still support GitHub Actions. While I can't find any announcement that they've dropped support, they have archived all of their repositories.

It may be prudent to find a replacement.

MaxBondABE commented 3 years ago

I have not found any service like Fuzzit for open source projects.

I would propose creating a second GHA workflow to implement fuzzing. The usage limits are fairly generous. We could run a task for up to 6 hours.

https://docs.github.com/en/actions/reference/usage-limits-billing-and-administration#usage-limits

My current plan is to implement grammatical fuzzing for the JSON example. If this works out, I can expand to the other example formats. We can run them in parallel for, say, 5 hours 45 minutes each, and then export any crashing examples as artifacts.

Let me know what your thoughts are.

Cheers, Max

dragostis commented 3 years ago

@MaxBondABE, I think any solution would be fine. I would be okay even with phasing out fuzzing as part of CI.

MaxBondABE commented 3 years ago

Okay. I'll submit a PR in a few hours to remove fuzzing, and I'll work on the GHA solution, which will take me a couple days to finish.

MarinPostma commented 3 years ago

@MaxBondABE What is the status of this issue? Do you need help with anything?