search

buildkite / pipeline-schema

A JSON schema for Buildkite’s pipeline file format
MIT License
31 stars 35 forks source link

tests don't seem to pass #82

Closed rogpeppe closed 2 weeks ago

rogpeppe commented 3 weeks ago

This is with respect to current tip (commit 0435a6c2dd4bd422488482ca7aacb6bac252365d).

I'm wanting to make a contribution to the schema, so the first thing I did was to try to verify that the tests pass. But I get test failures.

With npm test I see:

> test
> mocha *.test.js

  schema.json
    ✔ should validate block steps (56ms)
    ✔ should validate input steps
    1) should validate command steps
    ✔ should validate env blocks
    ✔ should validate blocks with extra properties
    ✔ should validate step groups
    ✔ should validate trigger steps
    ✔ should validate wait steps
    ✔ should validate notify
    ✔ should validate matrix

  9 passing (207ms)
  1 failing

  1) schema.json
       should validate command steps:
     AssertionError: expected 'data.steps[46] should NOT have additional properties, data.steps[46] should NOT have additional properties, data.steps[46] should NOT have additional properties, data.steps[46] should NOT have additional properties, data.steps[46] should be string, data.steps[46] should be equal to one of the allowed values, data.steps[46] should NOT have additional properties, data.steps[46] should NOT have additional properties, data.steps[46] should NOT have additional properties, data.steps[46] should NOT have additional properties, data.steps[46] should be string, data.steps[46] should be equal to one of the allowed values, data.steps[46] should NOT have additional properties, data.steps[46] should NOT have additional properties, data.steps[46].command should be object, data.steps[46] should be string, data.steps[46] should be equal to one of the allowed values, data.steps[46] should NOT have additional properties, data.steps[46] should NOT have additional properties, data.steps[46] should NOT have additional properties, data.steps[46] should NOT have additional properties, data.steps[46] should NOT have additional properties, data.steps[46] should NOT have additional properties, data.steps[46] should NOT have additional properties, data.steps[46] should NOT have additional properties, data.steps[46] should NOT have additional properties, data.steps[46] should NOT have additional properties, data.steps[46] should match some schema in anyOf, data.steps[47] should NOT have additional properties, data.steps[47] should NOT have additional properties, data.steps[47] should NOT have additional properties, data.steps[47] should NOT have additional properties, data.steps[47] should be string, data.steps[47] should be equal to one of the allowed values, data.steps[47] should NOT have additional properties, data.steps[47] should NOT have additional properties, data.steps[47] should NOT have additional properties, data.steps[47] should NOT have additional properties, data.steps[47] should be string, data.steps[47] should be equal to one of the allowed values, data.steps[47] should NOT have additional properties, data.steps[47] should NOT have additional properties, data.steps[47].command should be object, data.steps[47] should be string, data.steps[47] should be equal to one of the allowed values, data.steps[47] should NOT have additional properties, data.steps[47] should NOT have additional properties, data.steps[47] should NOT have additional properties, data.steps[47] should NOT have additional properties, data.steps[47] should NOT have additional properties, data.steps[47] should NOT have additional properties, data.steps[47] should NOT have additional properties, data.steps[47] should NOT have additional properties, data.steps[47] should NOT have additional properties, data.steps[47] should NOT have additional properties, data.steps[47] should match some schema in anyOf' to deeply equal undefined
      at validate (schema.test.js:16:48)
      at Context.<anonymous> (schema.test.js:28:5)
      at process.processImmediate (node:internal/timers:491:21)

With docker-compose, I see:

Creating test_tests_run ... 

> test
> mocha *.test.js

  schema.json
    ✔ should validate block steps (60ms)
    ✔ should validate input steps
    1) should validate command steps
    ✔ should validate env blocks
    ✔ should validate blocks with extra properties
    ✔ should validate step groups
    ✔ should validate trigger steps
    ✔ should validate wait steps
    ✔ should validate notify
    ✔ should validate matrix

  9 passing (230ms)
  1 failing

  1) schema.json
       should validate command steps:
     AssertionError: expected 'data.steps[46] should NOT have additional properties, data.steps[46] should NOT have additional properties, data.steps[46] should NOT have additional properties, data.steps[46] should NOT have additional properties, data.steps[46] should be string, data.steps[46] should be equal to one of the allowed values, data.steps[46] should NOT have additional properties, data.steps[46] should NOT have additional properties, data.steps[46] should NOT have additional properties, data.steps[46] should NOT have additional properties, data.steps[46] should be string, data.steps[46] should be equal to one of the allowed values, data.steps[46] should NOT have additional properties, data.steps[46] should NOT have additional properties, data.steps[46].command should be object, data.steps[46] should be string, data.steps[46] should be equal to one of the allowed values, data.steps[46] should NOT have additional properties, data.steps[46] should NOT have additional properties, data.steps[46] should NOT have additional properties, data.steps[46] should NOT have additional properties, data.steps[46] should NOT have additional properties, data.steps[46] should NOT have additional properties, data.steps[46] should NOT have additional properties, data.steps[46] should NOT have additional properties, data.steps[46] should NOT have additional properties, data.steps[46] should NOT have additional properties, data.steps[46] should match some schema in anyOf, data.steps[47] should NOT have additional properties, data.steps[47] should NOT have additional properties, data.steps[47] should NOT have additional properties, data.steps[47] should NOT have additional properties, data.steps[47] should be string, data.steps[47] should be equal to one of the allowed values, data.steps[47] should NOT have additional properties, data.steps[47] should NOT have additional properties, data.steps[47] should NOT have additional properties, data.steps[47] should NOT have additional properties, data.steps[47] should be string, data.steps[47] should be equal to one of the allowed values, data.steps[47] should NOT have additional properties, data.steps[47] should NOT have additional properties, data.steps[47].command should be object, data.steps[47] should be string, data.steps[47] should be equal to one of the allowed values, data.steps[47] should NOT have additional properties, data.steps[47] should NOT have additional properties, data.steps[47] should NOT have additional properties, data.steps[47] should NOT have additional properties, data.steps[47] should NOT have additional properties, data.steps[47] should NOT have additional properties, data.steps[47] should NOT have additional properties, data.steps[47] should NOT have additional properties, data.steps[47] should NOT have additional properties, data.steps[47] should NOT have additional properties, data.steps[47] should match some schema in anyOf' to deeply equal undefined
      at validate (schema.test.js:16:48)
      at Context.<anonymous> (schema.test.js:28:5)
      at process.processImmediate (node:internal/timers:476:21)

\ERROR: 1

What am I doing wrong?

rogpeppe commented 3 weeks ago

Investigating a bit further, it looks to me like this is a legit failure.

The test that's failing seems to be using this file as input.

When I manually check that file against the buildkit schema, I see validation errors.

I don't see cache as a valid field name in any of the definitions, but it's used in the final two steps of command.yml.

rogpeppe commented 3 weeks ago

Delving in still more, I see that the cache field test case was added in PR #78 (https://github.com/buildkite/pipeline-schema/pull/78/files#diff-9fcde326d127f74194f70e563bdf2c118c51b719c308f015b8eb0204a9a552fb).

That added cache as a definition inside #/definitions/commonOptions/cache but that definition is not referred to anywhere. I suspect it should be added as a property inside #/definitions/commandStep.

SorchaAbel commented 2 weeks ago

Thanks for fixing this :)