Open vchepkov opened 1 year ago
The problem is catalog-diff requests a catalog in pson
format so puppetserver cannot serialize the catalog in a way that preserves rich data, see
To request a "rich data" catalog you'll need to create the loaders on the client side and specify the rich data catalog mime type. If you're using the catalog v3 endpoint, then you can just use the builtin http client:
require 'puppet'
Puppet.initialize_settings
Puppet::Util::Log.newdestination(:console)
Puppet[:log_level] = :info
env = Puppet::Node::Environment.remote("production")
Puppet.push_context({
current_environment: env,
loaders: Puppet::Pops::Loaders.new(env, true),
rich_data: true
})
# Probably want to load previously saved facts using `Puppet::Node::Facts.from_data_hash` instead
facts = Puppet::Node::Facts.indirection.find(Puppet[:certname])
client = Puppet.runtime[:http]
session = client.create_session
service = session.route_to(:puppet)
response, catalog = service.post_catalog(Puppet[:certname], facts: facts, environment: env.name)
pp catalog.to_data_hash
For example, I see the Regexp
data type:
"resources"=>
[{"type"=>"Stage", "title"=>"main", "tags"=>["stage"], "exported"=>false, "kind"=>"compilable_type"},
{"type"=>"Class", "title"=>"Settings", "tags"=>["class", "settings"], "exported"=>false, "kind"=>"unknown"},
{"type"=>"Class", "title"=>"Main", "tags"=>["class"], "exported"=>false, "kind"=>"unknown", "parameters"=>{"name"=>"main"}},
{"type"=>"Node", "title"=>"default", "tags"=>["node", "default", "class"], "exported"=>false, "kind"=>"unknown"},
{"type"=>"Class", "title"=>"Abc", "tags"=>["class", "abc", "node", "default"], "exported"=>false, "kind"=>"unknown"},
{"type"=>"Class",
"title"=>"Abc::Bridges",
"tags"=>["class", "abc::bridges", "abc", "bridges", "node", "default"],
"exported"=>false,
"kind"=>"unknown",
"parameters"=>{"bridgenames"=>{"__ptype"=>"Regexp", "__pvalue"=>"^.*$"}}},
If you need to call the v4 catalog endpoint, then you'll need to pass the correct Accept
header, which can be resolved like:
format = Puppet::Network::FormatHandler.format_for(:rich_data_json)
headers['Accept'] = format.mime
And pass that to https://github.com/voxpupuli/puppet-catalog_diff/blob/952b5d468676936036f9823499f5543413450b56/lib/puppet/catalog-diff/compilecatalog.rb#L103
For the reference, and to answer @bastelfreak question, I use the following command options
$PUPPET catalog diff $PT_server/${PT_old_environment} \
$PT_server/${PT_new_environment} \
--certless --show_resource_diff --no-filter_old_env \
--render-as json --log_level warning >$TMPFILE1 2>$TMPFILE2
I tried to patch at least v4 catalog here:
https://github.com/vchepkov/puppet-catalog-diff/commit/7ab194aac19afba9d477e59d10c2e1540e1c8a8b
And It didn't help, @joshcooper , what did I miss?
2024-03-14T14:48:16.886Z INFO [qtp1628880978-39013] [puppetserver] Puppet Compiled catalog for t440.chepkov.lan in environment modules using the v4 catalog endpoint
2024-03-14T14:48:16.897Z ERROR [qtp1628880978-39013] [p.r.core] Internal Server Error: org.jruby.exceptions.RuntimeError: (PreformattedError) Evaluation Error: Class[Nftables::Bridges]['bridgenames'] contains a Regexp value. It will be converted to the String '/^br.+/'
at opt.puppetlabs.puppet.lib.ruby.vendor_ruby.puppet.pops.issue_reporter.assert_and_report(/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/issue_reporter.rb:60)
at RUBY.accept(/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/runtime3_support.rb:520)
at RUBY.accept(/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/validation.rb:216)
at RUBY.optionally_fail(/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/evaluator/runtime3_support.rb:44)
at RUBY.serialization_issue(/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/serialization/to_data_converter.rb:309)
at RUBY.unknown_to_string_with_warning(/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/serialization/to_data_converter.rb:206)
at opt.puppetlabs.puppet.lib.ruby.vendor_ruby.puppet.pops.serialization.to_data_converter.unknown_to_data(/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/serialization/to_data_converter.rb:195)
at opt.puppetlabs.puppet.lib.ruby.vendor_ruby.puppet.pops.serialization.to_data_converter.to_data(/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/serialization/to_data_converter.rb:120)
at opt.puppetlabs.puppet.lib.ruby.vendor_ruby.puppet.pops.serialization.to_data_converter.to_data(/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/serialization/to_data_converter.rb:109)
at opt.puppetlabs.puppet.lib.ruby.vendor_ruby.puppet.pops.serialization.to_data_converter.with(/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/serialization/to_data_converter.rb:171)
at opt.puppetlabs.puppet.lib.ruby.vendor_ruby.puppet.pops.serialization.to_data_converter.to_data(/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/serialization/to_data_converter.rb:109)
at org.jruby.RubyHash.each(org/jruby/RubyHash.java:1519)
at opt.puppetlabs.puppet.lib.ruby.vendor_ruby.puppet.pops.serialization.to_data_converter.to_data(/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/serialization/to_data_converter.rb:109)
at opt.puppetlabs.puppet.lib.ruby.vendor_ruby.puppet.pops.serialization.to_data_converter.with_recursive_guard(/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/serialization/to_data_converter.rb:189)
at opt.puppetlabs.puppet.lib.ruby.vendor_ruby.puppet.pops.serialization.to_data_converter.process(/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/serialization/to_data_converter.rb:158)
at opt.puppetlabs.puppet.lib.ruby.vendor_ruby.puppet.pops.serialization.to_data_converter.to_data(/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/serialization/to_data_converter.rb:106)
at opt.puppetlabs.puppet.lib.ruby.vendor_ruby.puppet.pops.serialization.to_data_converter.convert(/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/serialization/to_data_converter.rb:60)
at opt.puppetlabs.puppet.lib.ruby.vendor_ruby.puppet.pops.serialization.to_data_converter.convert(/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/pops/serialization/to_data_converter.rb:22)
at opt.puppetlabs.puppet.lib.ruby.vendor_ruby.puppet.resource.to_data_hash(/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource.rb:127)
at opt.puppetlabs.puppet.lib.ruby.vendor_ruby.puppet.resource.catalog.to_data_hash(/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:485)
at org.jruby.RubyArray.map(org/jruby/RubyArray.java:2667)
at opt.puppetlabs.puppet.lib.ruby.vendor_ruby.puppet.resource.catalog.to_data_hash(/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:485)
at RUBY.compile_catalog(uri:classloader:/puppetserver-lib/puppet/server/compiler.rb:115)
at RUBY.compile(uri:classloader:/puppetserver-lib/puppet/server/compiler.rb:29)
at RUBY.compileCatalog(uri:classloader:/puppetserver-lib/puppet/server/master.rb:101)
at opt.puppetlabs.puppet.lib.ruby.vendor_ruby.puppet.context.override(/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:62)
at opt.puppetlabs.puppet.lib.ruby.vendor_ruby.puppet.override(/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:289)
at RUBY.compileCatalog(uri:classloader:/puppetserver-lib/puppet/server/master.rb:100)
With puppetlabs/mysql module switching to Deferred function, I noticed that catalog_diff fails to evaluate the catalog. In puppetserver.log I observe the following error:
puppet 7.26.0 puppetserver 2021.7.5.17