search

RITlug / teleirc

Go implementation of a Telegram <=> IRC bridge for use with any IRC channel and Telegram group
https://docs.teleirc.com/
GNU General Public License v3.0
140 stars 45 forks source link

Make CI tests more demanding: run with testing config to evaluate successful start-up #98

Closed jwflory closed 4 years ago

jwflory commented 5 years ago

Summary

Ensure teleirc is able to start and stop successfully when running in the CI environment

Background

Recently, a breaking change accidentally made it to master even though our CI tests passed. It ran our code unit tests, but uncovered code got through and broke the application. Doing a full system start/stop is like a catch-all for anything that is not yet covered, to assert minimum functionality is there.

Details

I think the easiest way is to figure out a way to run CI builds with a testing config is the best way to do it. There are ways to encrypt variables for Travis CI in a repo, so we could do that with some of the API tokens. Needs more digging into.

Outcome

Making CI tests more reliable again

thenaterhood commented 5 years ago

Integration tests might be what you're looking for here. I'm on board with that.

On Sun, Dec 2, 2018, 10:29 Justin W. Flory notifications@github.com wrote:

Summary

Ensure teleirc is able to start and stop successfully when running in the CI environment Background

Recently, a breaking change accidentally made it to master even though our CI tests passed. It ran our code unit tests, but uncovered code got through and broke the application. Doing a full system start/stop is like a catch-all for anything that is not yet covered, to assert minimum functionality is there. Details

I think the easiest way is to figure out a way to run CI builds with a testing config is the best way to do it. There are ways to encrypt variables for Travis CI in a repo, so we could do that with some of the API tokens. Needs more digging into. Outcome

Making CI tests more reliable again

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/RITlug/teleirc/issues/98, or mute the thread https://github.com/notifications/unsubscribe-auth/ACdwxJb5RwKvMaDRVQMNuQ4AEkPMwDpvks5u0_HegaJpZM4Y9c-C .

Tjzabel commented 5 years ago

@jwflory do you have any more information about what the breaking change was that made it through the pipeline?

jwflory commented 5 years ago

@Tjzabel Method names were changed when the monster file was broken down into smaller classes, but some references to the new method names weren't updated. I haven't had a chance to dig into why tests did not catch this.

ct-martin commented 5 years ago

Just FYI, I will be trialing doing GitLab CI via the RITlug GitLab instance soon. You're welcome to use or not use it, but wanted to let you know it might be an option.

jwflory commented 4 years ago

Given the discussion today about unit tests and how old this issue is, I am closing this issue as out-of-scope. Our current discussions about testing and CI should cover this, especially with how the upstream Golang libraries handle config parsing.