voxpupuli / metadata-json-lint

Tool to check the validity of Puppet metadata.json files
Apache License 2.0
29 stars 27 forks source link

one test fails against ruby 2.7 when semantic_puppet is not present #114

Closed lelutin closed 3 years ago

lelutin commented 3 years ago

Hello,

I tried running the tests for metadata-json-lint agains ruby 2.7 (since it's the version in debian sid) without having semantic_puppet installed (I was basically trying to cut corners to avoid needing to create one more debian package in order to package metadata-json-lint into debian) and there was one test that was failing consistently:

RUBYLIB=. GEM_PATH=/<<PKGBUILDDIR>>/debian/metadata-json-lint/usr/share/rubygems-integration/all:/<<PKGBUILDDIR>>/debian/.debhelper/generated/_source/home/.local/share/gem/ruby/2.7.0:/var/lib/gems/2.7.0:/usr/local/lib/ruby/gems/2.7.0:/usr/lib/ruby/gems/2.7.0:/usr/lib/x86_64-linux-gnu/ruby/gems/2.7.0:/usr/share/rubygems-integration/2.7.0:/usr/share/rubygems-integration/all:/usr/lib/x86_64-linux-gnu/rubygems-integration/2.7.0 ruby2.7 -S rake -f debian/ruby-tests.rake
/usr/bin/ruby2.7 -I/usr/share/rubygems-integration/all/gems/rspec-support-3.9.3/lib:/usr/share/rubygems-integration/all/gems/rspec-core-3.9.2/lib /usr/share/rubygems-integration/all/gems/rspec-core-3.9.2/exe/rspec --pattern ./spec/\*\*/\*_spec.rb --format documentation
Could not find semantic_puppet gem, falling back to internal functionality. Version checks may be less robust.

MetadataJsonLint
  .validate_requirements!
    empty requirements
      is expected not to raise Exception
    with pe
      is expected not to raise Exception
    with invalid requirement
      is expected not to raise Exception (FAILED - 1)

MetadataJsonLint::Schema
  #schema
    is expected to be a kind of Hash
  #validate
    with empty hash
      is expected to be a kind of Array
      is expected to eq 7
      is expected to include {:field => "root", :message => "The file did not contain a required property of 'author'"}
    with minimal entries
      is expected to eq []
    with validation error on entry
      is expected to eq [{:field=>"summary", :message=>"The property 'summary' was not of a maximum string length of 144"}]
    with validation error on nested entry
      is expected to eq 1
      is expected to include {:field => "dependencies", :message => (a string matching /The property 'dependencies\/0\/name' value "in###id" did not a string matching the regex/)}
    with semver validation failure
      is expected to eq 1
      is expected to include {:field => "version", :message => (a string matching /The property 'version' must be a valid semantic version/)}

MetadataJsonLint::VersionRequirement
  .new
    is expected to be a kind of MetadataJsonLint::VersionRequirement
    is expected to be a kind of MetadataJsonLint::VersionRequirement
    is expected to raise ArgumentError
  #mixed_syntax?
    is expected to equal false
    is expected to equal false
    is expected to equal false
    is expected to equal false
    is expected to equal true
    is expected to equal true
    is expected to equal true
    is expected to equal true
    is expected to equal true
    is expected to equal true
    is expected to equal true
  #open_ended?
    is expected to equal false
    is expected to equal true
    is expected to equal true

Failures:

  1) MetadataJsonLint.validate_requirements! with invalid requirement is expected not to raise Exception
     Failure/Error: expect(described_class).to receive('error').with(:requirements, "Invalid 'version_requirement' field in metadata.json: Unparsable version range: \"a\"")

       (MetadataJsonLint).error(:requirements, "Invalid 'version_requirement' field in metadata.json: Unparsable version range: \"a\"")
           expected: 1 time with arguments: (:requirements, "Invalid 'version_requirement' field in metadata.json: Unparsable version range: \"a\"")
           received: 0 times
     # ./spec/metadata_json_lint_spec.rb:32:in `block (4 levels) in <top (required)>'

Finished in 0.04038 seconds (files took 0.22268 seconds to load)
30 examples, 1 failure

This might be somewhat expected since the warning does say that version parsing is maybe less accurate when running the test suite without semantic_puppet. The same test (all of them actually) does pass when semantic_puppet is installed as a build dependency.

If that test cannot pass without semantic_puppet, maybe it should be skipped if the library is not present? or maybe it has deeper implications that I haven't thought of..

(p.s. I'm going forward with packaging semantic_puppet since the tests all pass with it, so this can be viewed as a low priority issue -- I just thought I'd report this anyway since I have the test output)

ekohl commented 3 years ago

I recall that semantic_puppet was not a hard dependency because of the way it was vendored in Puppet's all-in-one packages and it created loading issues. It is best to consider it a hard dependency though.

lelutin commented 3 years ago

ok! then I guess nothing needs to be done here. the package uses ruby-semantic-puppet as a build dependency