reg-viz / reg-suit

:recycle: Visual Regression Testing tool
https://reg-viz.github.io/reg-suit
MIT License
1.18k stars 101 forks source link

I want to display multiple reports in one pull request or merge request #165

Open raigorai opened 5 years ago

raigorai commented 5 years ago

I want to display multiple reports in one pull request or merge request.

This is because a single repository manages multiple microservices. When a reg-suit run is run for each microservice, the report on the merge request is overwritten.

https://github.com/reg-viz/reg-suit/blob/master/packages/reg-notify-gitlab-plugin/src/use-cases.ts#L13 If I can set the COMMENT_MARK value arbitrarily, I can solve this problem.

Is there a plan to fix it?

Quramy commented 5 years ago

Hi. We're interested in your use case.

Perhaps this is not just gitlab plugin's issue.

I think we should introduce something like "project name" into reg-suit in order to use reg-suit with mono-repo.

For example:,

And the project.name property will be used from:

ra1028 commented 4 years ago

@Quramy Same request here. I'm currently using reg-suit for iOS application products. I have plans to develop multiple applications in mono-repo in the future and it would be useful to be able to display a separate report for each, IMO. I think to do that, in addition to the way you mention above, you'd need to be able to configure actualDir on a per-project basis.

fedevegili commented 4 years ago

@Quramy same use case here, having this project name setting would be amazing.

We don't even need this custom directory inside S3, we could use different buckets for each service inside the monorepo. However, the COMMENT_MARK is crucial. Being able to create a new github check would be nice as well, at the moment only one is displayed.

Do you accept PRs?

okmttdhr commented 2 years ago

For a workaround, we can refer to Chromatic;

Chromatic provides the following two options for the monorepo use cases. https://www.chromatic.com/docs/monorepos

  1. Combine multiple projects into a single Storybook
    • With --preserve-missing flag, Chromatic will pass the baselines forward and treat all missing stories as “preserved” without re-capturing them. Which means that we can only test a subset of stories.
    • If you are using a tool like Lerna or Nx, you can also capture only the project with changes
  2. Run Chromatic more than once in a second Chromatic project

These can also be applied to reg-suit.

  1. Combine multiple projects into a single Storybook
    • => In this case, since reg-suit doesn't have an option like --preserve-missing, it is likely to be necessary to capture Storybook of all projects every time. If it's OK for you, you can use this pattern (but if you have a huge number of projects in your monorepo, It doesn't seem to be scalable in principle)
  2. Run reg-suit more than once in a second Chromatic project
    • This is the pattern for running reg-suit run in each of all monorepo projects
    • => This is not possible with reg-suit because, as others have pointed out, only single comment will be created and overwritten (prCommentBehavior: new doesn't scale for monorepos with many projects)
dougfabris commented 2 years ago

@Quramy any news about a solution to run inside monorepos? We're really interested to run reg-suit in our projects!

tanmen commented 1 year ago

I too hope to take advantage of this use case.