search

alteryx / evalml

EvalML is an AutoML library written in python.
https://evalml.alteryx.com
BSD 3-Clause "New" or "Revised" License
774 stars 86 forks source link

CircleCI check on `master` intermittenty fails #743

Closed jeremyliweishih closed 4 years ago

jeremyliweishih commented 4 years ago

Example

CircleCI on master sometimes runs tests under the dep-update branch but will fail as the dep-update branch is used and deleted by the dependency GitHub action. I have disabled the action for now as this is known to fix the problem but the source of the problem is still unclear.

dsherry commented 4 years ago

Thanks @jeremyliweishih

We saw this a couple weeks ago too. What appeared to fix it at the time was to disable and re-enable all github actions.

I recently disabled the "evalml perf test" integration in our github. Mentioning because it could potentially be related.

Symptom: tests on the "commits" page in our github appear to run with branch dep-update and no author instead of branch master and the correct author. Github appears to be updating old branches with this.

Example: @christopherbunn merged this commit to master yesterday, and I saw all the checks were green then (except perf tests). Now, they're all red, as seen in this job.

Next steps: let's see what disabling the bot does this week and reassess. It would be nice to find a way to re-enable the bot. Perhaps taking a close look at the code would yield some insight. If this continues to happen we can consider contacting github (or circleci) support

dsherry commented 4 years ago

It does feel like this is a circleci problem, not a github problem. Because only the circleci test jobs were updated to point to the wrong branch, whereas the codecov jobs weren't updated.

I still don't see how the dependency update bot could have caused that...

@jeremyliweishih mentioned that he wrote to circleci support and will update when they write back.

dsherry commented 4 years ago

It appears our recent changes to the bot branch naming fixed this! Not sure how. Closing.