Testing examples in continuous integration

wachunga commented 6 years ago

My team's living styleguide is getting broken frequently without us noticing. In particular, examples in markdown files are falling out of date and end up broken. This is particularly concerning because those are supposed to be "best practice" snippets which are likely to be copy and pasted as a starting point for other devs.

This hasn't been an issue with storybook stories because the build will break in those cases, either because of snapshots changing or typescript will complain.

I would like to get our build failing in these cases so we catch this in CI. How would you recommend doing so? Is there documentation that I have overlooked?

I experimented with snapguistist, but there's no support for running those snapshots in CI yet and that wouldn't necessarily catch all classes of errors.

We're using react-docgen-typescript but I suppose this question is more general than that.

sapegin commented 6 years ago

Unfortunately I can't recommend anything because but that would be a good feature to have. Though I don't know if it should be in the core or as a third-party tool like Snapguidist.

wachunga commented 6 years ago

For the time being, I've externalized all my examples so they're proper components on their own (standard .tsx files). I use the updateExample config hook to load in the example files and format them a bit so they look nice in "view code", eg removing imports (that RSG doesn't need). Then for CI, I can run my usual linting etc, especially tsc --noEmit.

I like the inline examples but ultimately the CI and IDE experience was too poor for our needs.

To help make ensure the styleguide.config.js was all valid, I wrote another little script that iterates through sections and makes sure all the paths point to existing files:

function validateConfigSection(config) {
  if (config.content) {
    if (!fs.existsSync(config.content)) {
      throw new Error(`no style guide content found at path ${config.content} for section ${config.name}`);
    }
  }
  if (config.components) {
    const componentPaths = config.components();
    componentPaths.forEach(componentPath => {
      if (!fs.existsSync(componentPath)) {
        throw new Error(`no style guide component found at path ${componentPath} for section ${config.name}`);
      }
    });
  }

  if (config.sections) {
    config.sections.map(validateConfigSection);
  }
}

(This is probably less of a concern for those using component wildcards, which we hope to do eventually.)

ptbrowne commented 6 years ago

Hi,

I had the same need and eventually came up with a solution that is a bit hacky but works fairly well.

It enables me to write

testComponent('FolderName')

and the html result of the code sections in FolderName/Readme.md are snapshotted.

It is a bit hacky since I do not use the examples-loader from react-styleguidist, this means I have to do the automated requires "manually". Anyway, it works well for my needs. If anyone has a suggestion on how to improve it, I'm open :)

styleguidist / react-styleguidist

Testing examples in continuous integration #877