chromaui / addon-visual-tests

Visual Tests addon for Storybook
MIT License
33 stars 2 forks source link

Use `chromatic.config.json` rather than setting options in `main.js` #94

Closed tmeasday closed 1 year ago

tmeasday commented 1 year ago

Depends on https://github.com/chromaui/chromatic-cli/pull/814

Now you can only set configPath in main.js (the location of the configuration, which defaults to chromatic.config.json).

To QA

  1. Remove the chromatic.config.json, unset the main.js configPath option
  2. Check you can onboard
  3. Check it still works when your restart SB.
  4. Reset chromatic.config.json to only having the projectToken set (simulating someone who's configured Chromatic already)
  5. Do steps 2-3 again
  6. Make chromatic.config.json have invalid JSON, check for meaningful error.
  7. Set the configPath to something different (test.config.json), check for meaningful error when you start without the file existing
  8. Create test.config.json with just projectToken
  9. Do steps 2-3 again.

    📦 Published PR as canary version: 0.0.70--canary.94.0c85a5f.0
    :sparkles: Test out this PR locally via: ```bash npm install @chromaui/addon-visual-tests@0.0.70--canary.94.0c85a5f.0 # or yarn add @chromaui/addon-visual-tests@0.0.70--canary.94.0c85a5f.0 ```
linear[bot] commented 1 year ago
AP-3623 Add CLI support for a chromatic.js config file that allows multiple environments and configuration outside of .storybook/

Currently users must configure their chromatic cli inside of storybook configuration and through cli flags. This may be confusing as users might require different settings for local builds vs ci builds. We may be able to improve this by allowing users to define their configurations in a separate, type-safe config file. This should be an additional form of configuration, and allow backwards compatibility for existing setups using cli flags and ci configuration. It may need to allow both approaches to be mixed. **Multiple Environments:** With the addon-visual-tests having more users run builds outside of CI, configuration for the cli may need to vary based on the environment. Supporting multiple environments, through environment-suffixed names or programatic configuration would help optimize configuration for local builds. Related Slack thread: [https://chromaticqa.slack.com/archives/C051TQR6QLC/p1693198584200749](https://chromaticqa.slack.com/archives/C051TQR6QLC/p1693198584200749)

jonniebigodes commented 1 year ago

@tmeasday I'm still running some tests and share the findings tomorrow morning my time.

tmeasday commented 1 year ago

@weeksling

it removes the projectToken as soon as storybook starts.

That seems.. odd. Can you explain how I can reproduce that?

weeksling commented 1 year ago

@weeksling

it removes the projectToken as soon as storybook starts.

That seems.. odd. Can you explain how I can reproduce that?

@tmeasday change the chromatic.config.json to only have the projectToken (remove the projectId) and then start Storybook. It changes the file to be an empty object.

It looks like the first time it loads the configuration triggers the on change handler from my test

tmeasday commented 1 year ago

Fixed it, thanks @weeksling

jonniebigodes commented 1 year ago

@tmeasday so far, this is shaping up nicely; based on the QA instructions you left, it seems that one of the significant issues was already mentioned by @weeksling, and it seems fixed. However, during my testing, I noticed some things that require attention from the CLI pr.

Here's a bit of an overview of what I uncovered and the steps taken:

tmeasday commented 1 year ago

Hey @jonniebigodes that's just because I forgot to add zip as an allowed configuration option! I guess it doesn't error on unknown keys, I might add that.