screenshotbot / screenshotbot-oss

A Screenshot Testing service to tie with your existing Android, iOS and Web screenshot tests
https://screenshotbot.io
Mozilla Public License 2.0
238 stars 13 forks source link

How to promote test runs? #21

Closed SamCosta1 closed 2 years ago

SamCosta1 commented 2 years ago

More of a question than an issue.

But how can I promote a run via the cli? I'm currently seeing a list of unpromoted runs in my dashboard but can't see any way to promote one

Screenshot 2022-06-13 at 17 49 33

However there are some errors in the promotion log so possibly this is to do with the fact I'm using bitbucket?

  --BEGIN-LOGS--
ERROR: No github installation id avaialble
INFO: Repo is of type: SCREENSHOTBOT/PRO/BITBUCKET/PLUGIN:BITBUCKET-REPO
ERROR: No github installation id avaialble
INFO: Repo is of type: SCREENSHOTBOT/PRO/BITBUCKET/PLUGIN:BITBUCKET-REPO
ERROR: No github installation id avaialble
INFO: Repo is of type: SCREENSHOTBOT/PRO/BITBUCKET/PLUGIN:BITBUCKET-REPO
ERROR: No github installation id avaialble
INFO: Repo is of type: SCREENSHOTBOT/PRO/BITBUCKET/PLUGIN:BITBUCKET-REPO

  --END-LOGS--
tdrhq commented 2 years ago

Let me take a look, it does look like something crashed in the Bitbucket logic

tdrhq commented 2 years ago

For promotion, promotion only happens on the master branch. And there are a bunch of other checks to make sure that a developer doesn't accidentally run a CI script that causes a false promotion.

In your case it looks like the runs on e4418358 detected that the master branch had commit 4dee963a. If e4418 was indeed already committed to master, then this is likely because Bitrise doesn't fetch all the remote branches (I don't exactly remember Bitrise's behaviour, it's been some time since I looked at it). Try adding a git fetch origin master as a CI step before calling the screenshotbot CLI.

(Separately, I fixed crashes with your pull request notifications. You should now have build statuses that say that the "Base SHA wasn't available for comparison. In those cases it detected the merge base as 14dd4f09. If that is incorrect, it might also be related to not doing the git fetch origin master step)

SamCosta1 commented 2 years ago

I think the issue was the way I was running the test runs, I was doing them locally and in a strange order.

I cleaned a few things up and this is now working! Thank you!