search

goss-org / goss

Quick and Easy server testing/validation
https://goss.rocks
Apache License 2.0
5.5k stars 470 forks source link

Improve CI pipeline and integration tests #911

Open dklimpel opened 1 month ago

dklimpel commented 1 month ago

I have opened a few PR to move from Travis CI to Github and update integration tests.

The following proposal for processing the PR.

Independent of this, new feature:

Independent of this, fix docs:

  1. 900

  2. 904

  3. 919

bring back linter:

  1. 899

  2. ~later some PR fix linting issues~

  3. 922

  4. ~bring back linter for windows~

update integration test docker images:

  1. 907

  2. 902

  3. 906

  4. 908

  5. new PR to remove old images

  6. 901

  7. new PR to uses new docker images from ghcr (incl. new MD5 checksums to bypass build in CI)
  8. decide when to build new docker test images

After that it is possible to move from Travis CI to Github Actions.

It may be useful to provide all CIs with a uniform wording and an optimized sequence at the end.

aelsabbahy commented 1 month ago

You've been busy, love it!

I'll go through these PRs and start reviewing/merging the ones with least dependencies.

Two questions regarding integration-tests:

  1. Did you ever find a fix for the systemd in docker thing?
  2. I haven't looked at the PRs yet, do they leverage the make targets?
    • If not, I want to make sure it's reproducible locally outside of CI if possible.
    • There's also act - written by a former colleague of mine, one of the best engineers I had the pleasure of working with. I've never personally used it.
dklimpel commented 1 month ago

Did you ever find a fix for the systemd in docker thing?

The topic is very annoying. Docker is simply not made for running multiple processes and monitoring them from a separate process (systemd)- there are solutions to start multiple processes in a container. But that won't help, because the container is used to simulate an operating system. The whole thing can only be solved by calling docker run with the correct parameters.

I haven't looked at the PRs yet, do they leverage the make targets?

All these changes run with Travis CI, Github workflow and locally on my ubuntu machine (make test-all)

One issue remains open from my point of view. The test in which runlevel a service runs is not compatible with current linux versions. My knowledge of go is not great enough to investigate this.

aelsabbahy commented 1 month ago

@dklimpel random question, are you on gophers slack by any chance?

aelsabbahy commented 1 month ago

Between #900 and #904 is there a dependency or order to merge them. Both have failing builds currently.

After those are merged, then the long awaited journey to move off of Travis begins.

dklimpel commented 1 month ago

Both have different failing jobs. And both fix different jobs. I would start with #900 and then merge main to #904. Then should #904 not failing anymore.

aelsabbahy commented 1 month ago

900 merged, #904 updated

aelsabbahy commented 1 month ago

One issue remains open from my point of view. The test in which runlevel a service runs is not compatible with current linux versions. My knowledge of go is not great enough to investigate this.

Which Linux (branch) is failing, I can checkout that brance and investigate next week. Wonder if it's a Goss bug, or working as intended.

dklimpel commented 1 month ago

Which Linux (branch) is failing, I can checkout that brance and investigate next week. Wonder if it's a Goss bug, or working as intended.

It is a problem with Debian and Ubuntu:

Also on my local Ubuntu machine.

aelsabbahy commented 1 month ago

Okay, thanks. I'll check them both out.

Which PRs are next for documentation?

https://github.com/goss-org/goss/pull/919?

dklimpel commented 1 month ago

I have tried to put it into a sorted list above.

904 fix the linting issues. After that there should be a valid documentation. #919 fixing for upcoming changes that the documentation pipeline becomes triggered.

dklimpel commented 1 month ago

GitHub creates a workflow when push to master automatically. https://github.com/goss-org/goss/actions/workflows/pages/pages-build-deployment

IMHO this is failing because this is a default job with Jekyll theme. Probably this can be disabled in project settings. https://github.com/goos-org/goss/settings/pages - Change to source "GitHub Action". Readthedocs should not need this, because it is working by triggers, I think. GitHub pages is not used here.

aelsabbahy commented 1 month ago

Changed, I guess next PR to be merged will validate this?

Also, please continue to use this issue to let me know the next PR in the chain. I find it a lot easier to track on here.

This is an amazing level of work by the way, much appreciated. It's something I've wanted for a long time. Unfortunately, due to limited time I never got around to it, my focus tends to be:

  1. Bugs/security findings
  2. Features
  3. Everything else (refactor, CI, etc..)
dklimpel commented 1 month ago

Changed, I guess next PR to be merged will validate this?

Yes, it is.

My suggestion for the next steps.

dklimpel commented 1 month ago

@dklimpel random question, are you on gophers slack by any chance?

Unfortunately not.

aelsabbahy commented 1 month ago

Hey @dklimpel , if you don't mind. Let me know on here the next PR that's ready and I'll review.

This is an awesome amount of work you put it, it's greatly appreciated!

dklimpel commented 1 month ago

To improve the code:

aelsabbahy commented 2 weeks ago

Holding off on merging more PRs until Travis oss credits are replenished.

Don't want CI to get in a broken state with unclear traceability on what caused it.

This is an awesome amount of work. Can't thank you enough for taking the time to do this.

dklimpel commented 2 weeks ago

Holding off on merging more PRs until Travis oss credits are replenished.

Ok.

dklimpel commented 2 weeks ago

I think #928 can help. This enables unit tests with GitHub Pipeline.

aelsabbahy commented 5 days ago

Using this as a central coordination point. What PRs are ready for merge, so I can start going through them.

dklimpel commented 2 days ago

I would recommend finalizing a few topics before we take a look at the docker images.