Setup Travis crons

[machour]

As briefly discussed on slack, it's becoming harder and harder to understand build failures in Yii 3 repos, as packages are inter-dependent and a tiny change in A can brake B without us noticing for days.

As a counter measure, could we setup Travis cronjobs as suggested by @rob006 on all repositories to run like every 2 days? (still unsure about the right periodicity)

[samdark]

Seems not to be a problem now.

[rob006]

@samdark IMO it is needed more than ever. With current fragmentation of packages it may be easy to break some package by minor BC break in used dependency (so change in package A may break package B).

[samdark]

What package should we test as integration one?

[rob006]

This is not about integration tests. This is about breaking package tests by some external changes. Every package should have tests run every X days to ensure that they're still working fine.

[rugabarbo]

Where will we send notifications?

Travis supports the following options:

• Slack
• Email
• webhooks
• IRC
• Campfire
• Flowdock
• HipChat
• Pushover
• OpsGenie

It seems to me that the best idea for group alerts in our case is Slack. Travis documentation has great example of separating alerts based on the build result. This allows you to receive notifications of successful builds in one channel, and notifications of failed builds in another:

notifications:
  slack:
    - rooms:
        - <account>:<token>#failures
      on_success: never
      on_failure: always
      template:
        - "%{repository_slug} (%{commit}) : %{message}"
        - "Build details: %{build_url}"
    - rooms:
        - <account>:<token>#successes
      on_success: always
      on_failure: never
      template:
        - "%{repository_slug} (%{commit}) : %{message}"
        - "Build details: %{build_url}"

[samdark]

Yes, Slack should be fine.

[rugabarbo]

@samdark can you add two new slack rooms for these purposes? We will invite everyone who is interested in receiving notifications to these rooms.

[samdark]

Added build-successes and build-failures.

[rugabarbo]

@samdark now we need encrypted credentials for Slack integration, which we can use in public repositories. Set up a new Travis CI integration and encrypt the tokens of each room.

I need encrypted credentials for both rooms. Our public configuration will look something like this:

notifications:
    slack:
        - rooms:
              - secure: "ABC5OwLpwB7L6Ca...."
          on_success: never
          on_failure: always
          template:
              - "%{repository_slug} (%{commit}) : %{message}"
              - "Build details: %{build_url}"
        - rooms:
              - secure: "ABC5OwLpwB7L6Ca...."
          on_success: always
          on_failure: never
          template:
              - "%{repository_slug} (%{commit}) : %{message}"
              - "Build details: %{build_url}"

You see here the encrypted string "ABC5OwLpwB7L6Ca....", which I need for adjusting travis configs in yii public repositories.

[samdark]

Your workspace has reached the integration limit

:(

[rugabarbo]

Hmm...

Do we really need all these apps? How to understand which of this can be safely disabled? What is not used? Is it real to understand?

[samdark]

Oh. Wow. I have no idea why all these are enabled :) Thanks for pointing it.

[samdark]

Not sure I've got the tokens correctly but it should be so.

[rugabarbo]

Great :)

Now... for a test we'll take two repositories – one with the last successful build and one with a failed one. 1) https://travis-ci.org/yiisoft/di 2) https://travis-ci.org/yiisoft/yii-captcha

In each I'll create a PR with a configuration of notifications. Next, we'll add Travis cronjobs for them and make sure that everything works as intended.

[samdark]

Do you need these merged?

[rugabarbo]

Yes, merge both PRs to the master branch in both repositories.

[samdark]

Done.

[rugabarbo]

Now open two pages: 1) https://travis-ci.com/yiisoft/di/settings 2) https://travis-ci.com/yiisoft/yii-captcha/settings

Find "Cron Jobs" section there.

Add the following schedule for both repositories:

[samdark]

Scheduled.

[rugabarbo]

Hmm ... something strange happened. In Travis, the build history for these repositories has disappeared. 1) https://travis-ci.com/yiisoft/di/builds 2) https://travis-ci.com/yiisoft/yii-captcha/builds

I no longer see a single build on these pages :(

[samdark]

These are at .org (yes, messy): https://travis-ci.org/yiisoft/di/builds

[samdark]

I've set up there as well.

[rugabarbo]

Damn, these Travis domains are some kind of hell :) Now we will wait for a builds on schedule.

[samdark]

I'm thinking more and more about CircleCI...

[rugabarbo]

Two builds ended more than 20 minutes ago:

1. https://travis-ci.org/yiisoft/di/builds/597764297
2. https://travis-ci.org/yiisoft/yii-captcha/builds/597764290

But there are no notifications in Slack... So something went wrong. I'll try to understand what the problem is.

I'm thinking more and more about CircleCI...

With the exception of domain confusion, Travis seems to work well :) There are no other objective reasons for changing it...

[samdark]

Well, https://circleci.com/circleci-versus-travisci/

[samdark]

[rugabarbo]

Well, https://circleci.com/circleci-versus-travisci/

This is just marketing... We should look at what is really inconvenient for us now in Travis, and not what CircleCI promises in its comparisons :)

[samdark]

The issue with notifications is likely connected with the fact that encrypted tokens are tied to concrete repository (this one). Repository is specified during the generation process. It doesn't seems we can omit it (but there could be a way I haven't seen).

[rugabarbo]

I just programmatically encrypted Slack token without using Travis CI client. Then created PR using this token: https://github.com/yiisoft/yii-captcha/pull/23

Build failed: https://travis-ci.org/yiisoft/yii-captcha/builds/598125538 Notification came to Slack:

Thus, the process of generating encrypted tokens for 80 repositories can be automated.

[samdark]

Is it still per-repository token?

[rugabarbo]

Is it still per-repository token?

Yes. It is :) The token can only be generated using the name of the repository.

The process of setting schedules can also be automated: https://developer.travis-ci.org/resource/cron#create

It seems that it’s possible to write a small script that will do everything for us :)

I'll think a little more about it...

[rugabarbo]

Guys, for clarity: I’m going to first deal with task #27, and then implement a console command that can generate Travis cronjobs and encrypted Slack tokens for integration.

[rugabarbo]

@samdark

1. Merge #57
2. Update yii-dev-tool locally
3. Go to Slack and copy Travis CI token
4. Place the token in file yii-dev-tool/config/travis/slack.local.php (use slack.local.php.example as example)
5. Prepare the repositories for pushing new things (use updated clean master branch)
6. Run ./yii-dev travis/update-slack-config --verbose
7. Push the changes to the repositories

[samdark]

@rugabarbo done.

[samdark]

So now we have notifications. What's left if setting up cronjobs.

[samdark]

Thank you, @rugabarbo.

[rugabarbo]

I have installed a cronjob for all repositories using command ./yii-dev travis/ensure-cronjob (see PR #58).

I had to install both on travis-ci.COM and travis-ci.ORG, because it’s not possible to determine exactly which domain each repository is active on 😞 Mess...

[samdark]

Everything seems to be in place now. Closing.

[samdark]

Thanks, @rugabarbo

[rugabarbo]

@samdark reopen, please.

Builds of repositories, which are located only on domain travis-ci.COM, are launched by cron, but notifications do not come to Slack...

We need to think about why this is happening and fix it.

See details (carefully, there will be a big table and many pictures):

Package | travis-ci.COM | travis-ci.ORG | Slack Notification ---|---|---|--- docs | [](https://travis-ci.com/yiisoft/docs) | [](https://travis-ci.org/yiisoft/docs) | di | [](https://travis-ci.com/yiisoft/di) | [](https://travis-ci.org/yiisoft/di) | ✔ injector | [](https://travis-ci.com/yiisoft/injector) | [](https://travis-ci.org/yiisoft/injector) | factory | [](https://travis-ci.com/yiisoft/factory) | [](https://travis-ci.org/yiisoft/factory) | access | [](https://travis-ci.com/yiisoft/access) | [](https://travis-ci.org/yiisoft/access) | event-dispatcher | [](https://travis-ci.com/yiisoft/event-dispatcher) | [](https://travis-ci.org/yiisoft/event-dispatcher) | ✔ security | [](https://travis-ci.com/yiisoft/security) | [](https://travis-ci.org/yiisoft/security) | data | [](https://travis-ci.com/yiisoft/data) | [](https://travis-ci.org/yiisoft/data) | profiler | [](https://travis-ci.com/yiisoft/profiler) | [](https://travis-ci.org/yiisoft/profiler) | aliases | [](https://travis-ci.com/yiisoft/aliases) | [](https://travis-ci.org/yiisoft/aliases) | serializer | [](https://travis-ci.com/yiisoft/serializer) | [](https://travis-ci.org/yiisoft/serializer) | network-utilities | [](https://travis-ci.com/yiisoft/network-utilities) | [](https://travis-ci.org/yiisoft/network-utilities) | auth | [](https://travis-ci.com/yiisoft/auth) | [](https://travis-ci.org/yiisoft/auth) | cache | [](https://travis-ci.com/yiisoft/cache) | [](https://travis-ci.org/yiisoft/cache) | ✔ cache-apcu | [](https://travis-ci.com/yiisoft/cache-apcu) | [](https://travis-ci.org/yiisoft/cache-apcu) | cache-wincache | [](https://travis-ci.com/yiisoft/cache-wincache) | [](https://travis-ci.org/yiisoft/cache-wincache) | cache-file | [](https://travis-ci.com/yiisoft/cache-file) | [](https://travis-ci.org/yiisoft/cache-file) | cache-db | [](https://travis-ci.com/yiisoft/cache-db) | [](https://travis-ci.org/yiisoft/cache-db) | rbac | [](https://travis-ci.com/yiisoft/rbac) | [](https://travis-ci.org/yiisoft/rbac) | ✔ rbac-db | [](https://travis-ci.com/yiisoft/rbac-db) | [](https://travis-ci.org/yiisoft/rbac-db) | rbac-php | [](https://travis-ci.com/yiisoft/rbac-php) | [](https://travis-ci.org/yiisoft/rbac-php) | log | [](https://travis-ci.com/yiisoft/log) | [](https://travis-ci.org/yiisoft/log) | ✔ log-target-db | [](https://travis-ci.com/yiisoft/log-target-db) | [](https://travis-ci.org/yiisoft/log-target-db) | ✔ log-target-email | [](https://travis-ci.com/yiisoft/log-target-email) | [](https://travis-ci.org/yiisoft/log-target-email) | ✔ log-target-file | [](https://travis-ci.com/yiisoft/log-target-file) | [](https://travis-ci.org/yiisoft/log-target-file) | ✔ log-target-syslog | [](https://travis-ci.com/yiisoft/log-target-syslog) | [](https://travis-ci.org/yiisoft/log-target-syslog) | ✔ i18n | [](https://travis-ci.com/yiisoft/i18n) | [](https://travis-ci.org/yiisoft/i18n) | ✔ i18n-message-php | [](https://travis-ci.com/yiisoft/i18n-message-php) | [](https://travis-ci.org/yiisoft/i18n-message-php) | ✔ i18n-message-gettext | [](https://travis-ci.com/yiisoft/i18n-message-gettext) | [](https://travis-ci.org/yiisoft/i18n-message-gettext) | ✔ i18n-formatter-intl | [](https://travis-ci.com/yiisoft/i18n-formatter-intl) | [](https://travis-ci.org/yiisoft/i18n-formatter-intl) | ✔ yii-queue | [](https://travis-ci.com/yiisoft/yii-queue) | [](https://travis-ci.org/yiisoft/yii-queue) | ✔ yii-queue-interop | [](https://travis-ci.com/yiisoft/yii-queue-interop) | [](https://travis-ci.org/yiisoft/yii-queue-interop) | mutex | [](https://travis-ci.com/yiisoft/mutex) | [](https://travis-ci.org/yiisoft/mutex) | mutex-file | [](https://travis-ci.com/yiisoft/mutex-file) | [](https://travis-ci.org/yiisoft/mutex-file) | ✔ mutex-db-pgsql | [](https://travis-ci.com/yiisoft/mutex-db-pgsql) | [](https://travis-ci.org/yiisoft/mutex-db-pgsql) | ✔ mutex-db-oracle | [](https://travis-ci.com/yiisoft/mutex-db-oracle) | [](https://travis-ci.org/yiisoft/mutex-db-oracle) | ✔ mutex-db-mysql | [](https://travis-ci.com/yiisoft/mutex-db-mysql) | [](https://travis-ci.org/yiisoft/mutex-db-mysql) | ✔ mailer | [](https://travis-ci.com/yiisoft/mailer) | [](https://travis-ci.org/yiisoft/mailer) | ✔ mailer-swiftmailer | [](https://travis-ci.com/yiisoft/mailer-swiftmailer) | [](https://travis-ci.org/yiisoft/mailer-swiftmailer) | ✔ arrays | [](https://travis-ci.com/yiisoft/arrays) | [](https://travis-ci.org/yiisoft/arrays) | ✔ strings | [](https://travis-ci.com/yiisoft/strings) | [](https://travis-ci.org/yiisoft/strings) | ✔ files | [](https://travis-ci.com/yiisoft/files) | [](https://travis-ci.org/yiisoft/files) | var-dumper | [](https://travis-ci.com/yiisoft/var-dumper) | [](https://travis-ci.org/yiisoft/var-dumper) | html | [](https://travis-ci.com/yiisoft/html) | [](https://travis-ci.org/yiisoft/html) | json | [](https://travis-ci.com/yiisoft/json) | [](https://travis-ci.org/yiisoft/json) | yii-console | [](https://travis-ci.com/yiisoft/yii-console) | [](https://travis-ci.org/yiisoft/yii-console) | ✔ yii-rest | [](https://travis-ci.com/yiisoft/yii-rest) | [](https://travis-ci.org/yiisoft/yii-rest) | ✔ db | [](https://travis-ci.com/yiisoft/db) | [](https://travis-ci.org/yiisoft/db) | ✔ db-mysql | [](https://travis-ci.com/yiisoft/db-mysql) | [](https://travis-ci.org/yiisoft/db-mysql) | ✔ db-pgsql | [](https://travis-ci.com/yiisoft/db-pgsql) | [](https://travis-ci.org/yiisoft/db-pgsql) | ✔ db-sqlite | [](https://travis-ci.com/yiisoft/db-sqlite) | [](https://travis-ci.org/yiisoft/db-sqlite) | ✔ db-mssql | [](https://travis-ci.com/yiisoft/db-mssql) | [](https://travis-ci.org/yiisoft/db-mssql) | db-oracle | [](https://travis-ci.com/yiisoft/db-oracle) | [](https://travis-ci.org/yiisoft/db-oracle) | db-mongodb | [](https://travis-ci.com/yiisoft/db-mongodb) | [](https://travis-ci.org/yiisoft/db-mongodb) | ✔ active-record | [](https://travis-ci.com/yiisoft/active-record) | [](https://travis-ci.org/yiisoft/active-record) | ✔ yii-cycle | [](https://travis-ci.com/yiisoft/yii-cycle) | [](https://travis-ci.org/yiisoft/yii-cycle) | router | [](https://travis-ci.com/yiisoft/router) | [](https://travis-ci.org/yiisoft/router) | router-fastroute | [](https://travis-ci.com/yiisoft/router-fastroute) | [](https://travis-ci.org/yiisoft/router-fastroute) | yii-web | [](https://travis-ci.com/yiisoft/yii-web) | [](https://travis-ci.org/yiisoft/yii-web) | ✔ view | [](https://travis-ci.com/yiisoft/view) | [](https://travis-ci.org/yiisoft/view) | ✔ yii-jquery | [](https://travis-ci.com/yiisoft/yii-jquery) | [](https://travis-ci.org/yiisoft/yii-jquery) | ✔ yii-masked-input | [](https://travis-ci.com/yiisoft/yii-masked-input) | [](https://travis-ci.org/yiisoft/yii-masked-input) | ✔ yii-dataview | [](https://travis-ci.com/yiisoft/yii-dataview) | [](https://travis-ci.org/yiisoft/yii-dataview) | yii-debug | [](https://travis-ci.com/yiisoft/yii-debug) | [](https://travis-ci.org/yiisoft/yii-debug) | ✔ yii-gii | [](https://travis-ci.com/yiisoft/yii-gii) | [](https://travis-ci.org/yiisoft/yii-gii) | ✔ yii-bootstrap4 | [](https://travis-ci.com/yiisoft/yii-bootstrap4) | [](https://travis-ci.org/yiisoft/yii-bootstrap4) | ✔ yii-captcha | [](https://travis-ci.com/yiisoft/yii-captcha) | [](https://travis-ci.org/yiisoft/yii-captcha) | ✔ yii-auth-client | [](https://travis-ci.com/yiisoft/yii-auth-client) | [](https://travis-ci.org/yiisoft/yii-auth-client) | ✔ yii-base-api | [](https://travis-ci.com/yiisoft/yii-base-api) | [](https://travis-ci.org/yiisoft/yii-base-api) | ✔ yii-base-web | [](https://travis-ci.com/yiisoft/yii-base-web) | [](https://travis-ci.org/yiisoft/yii-base-web) | ✔ yii-project-template | [](https://travis-ci.com/yiisoft/yii-project-template) | [](https://travis-ci.org/yiisoft/yii-project-template) | ✔ yii-demo | [](https://travis-ci.com/yiisoft/yii-demo) | [](https://travis-ci.org/yiisoft/yii-demo) | yii-docker | [](https://travis-ci.com/yiisoft/yii-docker) | [](https://travis-ci.org/yiisoft/yii-docker) | validator | [](https://travis-ci.com/yiisoft/validator) | [](https://travis-ci.org/yiisoft/validator) | ✔ friendly-exception | [](https://travis-ci.com/yiisoft/friendly-exception) | [](https://travis-ci.org/yiisoft/friendly-exception) | template | [](https://travis-ci.com/yiisoft/template) | [](https://travis-ci.org/yiisoft/template) | requirements | [](https://travis-ci.com/yiisoft/requirements) | [](https://travis-ci.org/yiisoft/requirements) |

[samdark]

Arrgh. That's a mess with .com and .org :(

[rugabarbo]

Keep calm :)

I have found a reason. For travis-ci.COM, we need to get the keys differently during Slack token encryption: https://docs.travis-ci.com/user/encryption-keys/#obtaining-the-public-keys

I'll fix it.

[samdark]

I can migrate everything to .com so it will be consistent. They finally have migration procedure available.

[rugabarbo]

I can write a small console command for automatic migration. Travis API provides the ability to start migration: https://developer.travis-ci.com/resource/repository#migrate

How difficult is it to migrate everything manually? Is it possible to migrate everything with one click? If it’s difficult to migrate manually, it will only take me 10 minutes to write a command and start the migration using Travis API.

[samdark]

How difficult is it to migrate everything manually? Is it possible to migrate everything with one click?

Yes.

[samdark]

Migrated.

[rugabarbo]

@samdark

1. Merge #59
2. Update yii-dev-tool locally
3. Generate and copy Travis API token for travis-ci.COM
4. Place the token in file yii-dev-tool/config/travis/api.local.php (use api.local.php.example as example)
5. Prepare the repositories for pushing new things (use updated clean master branch)
6. Run ./yii-dev travis/update-slack-config --verbose
7. Push the changes to the repositories

Pay attention, now in steps 3 and 4 you need tokens specifically for Travis API, not for Slack. Leave Slack token the same as it was!

[samdark]

All done but it seems there are problems:

• .org builds are still triggered. There are stale webhooks from very old travis-ci.org in each repository in yiisoft. https://notify.travis-ci.org/
• .com builds seems to be triggered properly but aren't ending up in Slack

[samdark]

I'll handle .org issue separately. Slack works well. Thank you again for preparing all that.