augeas is broken on OSX

[h4l]

Describe the bug PDK ships libaugeas which is built incorrectly on OSX, augeas functionality does not work as a result.

To Reproduce Two Puppet and Ruby programs to demonstrate this:

# augtest.pp
augeas("msg1=abc\nmsg2=def\n", 'Properties.lns', ['dump-xml /store'])

# augtest.rb
require 'augeas'
aug = Augeas.open(nil, nil, Augeas::NO_MODL_AUTOLOAD)

res, out = aug.srun('set /input "msg1=abc\nmsg2=def\n"')
raise "#{aug.error[:message]}: #{aug.error[:details]}" if res < 0

res, out = aug.srun('store "Simplelines.lns" /input /tree')
raise "#{aug.error[:message]}: #{aug.error[:details]}" if res < 0

res, out = aug.srun('dump-xml /tree')
raise "#{aug.error[:message]}: #{aug.error[:details]}" if res < 0

$ which ruby
/opt/puppetlabs/pdk/private/ruby/2.7.3/bin/ruby

I've got a separate copy of augeas from Homebrew which does work.

$ brew info augeas
augeas: stable 1.12.0 (bottled), HEAD
Configuration editing tool and API
https://augeas.net/
/usr/local/Cellar/augeas/1.12.0 (473 files, 3.5MB) *
  Poured from bottle on 2020-11-01 at 14:39:11
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/augeas.rb
License: LGPL-2.1
==> Dependencies
Build: autoconf ✔, automake ✘, bison ✘, libtool ✔, pkg-config ✔
Required: readline ✔
==> Options
--HEAD
        Install HEAD version
==> Caveats
Lenses have been installed to:
  /usr/local/share/augeas/lenses/dist
==> Analytics
install: 3,967 (30 days), 16,041 (90 days), 54,244 (365 days)
install-on-request: 1,164 (30 days), 5,337 (90 days), 10,982 (365 days)
build-error: 8 (30 days)

As already reported in #743, PDK does not ship with any augeas lenses:

$ ruby augtest.rb
Traceback (most recent call last):
augtest.rb:8:in `<main>': Lens not found: Can not find lens Simplelines.lns (RuntimeError)

If we use the lenses from homebrew's augeas we find PDK's augeas fails to parse the lens files:

$ AUGEAS_LENS_LIB=/usr/local/share/augeas/lenses/dist ruby augtest.rb
Traceback (most recent call last):
augtest.rb:8:in `<main>': Syntax error in lens definition: /usr/local/share/augeas/lenses/dist/util.aug:115.2-123.51:Failed to compile comment_multiline (RuntimeError)
/usr/local/share/augeas/lenses/dist/util.aug:116.20-.69:exception: Regular expression subtraction 'r1 - r2' failed
    r1: /[^     \r
].*[^   \r
]|[^    \r
]/
    r2: /.*\\*/.*/

/usr/local/share/augeas/lenses/dist/simplelines.aug:28.21-.32:Could not load module Util for Util.indent
/usr/local/share/augeas/lenses/dist/simplelines.aug:28.21-.32:Undefined variable Util.indent
/usr/local/share/augeas/lenses/dist/simplelines.aug:33.10-.21:Undefined variable Util.empty
/usr/local/share/augeas/lenses/dist/simplelines.aug:48.20-.23:Undefined variable lns

If we force the use of libaugeas from homebrew instead of PDK it works as expected:

$ DYLD_LIBRARY_PATH=/usr/local/Cellar/augeas/1.12.0/lib AUGEAS_LENS_LIB=/usr/local/share/augeas/lenses/dist ruby augtest.rb
<augeas match="/tree">
  <node label="tree" path="/tree">
    <node label="1">
      <value>msg1=abc</value>
    </node>
    <node label="2">
      <value>msg2=def</value>
    </node>
  </node>
</augeas>

Same story when using augeas via puppet:

$ which puppet
/opt/puppetlabs/pdk/private/puppet/ruby/2.7.0/bin/puppet

$ puppet apply --modulepath modules augtest.pp
<augeas match="/store">
  <node label="store" path="/store"/>
</augeas>
Error: Evaluation Error: Error while evaluating a Function Call, augeas(): Failed to apply changes with lens Properties.lns:  (file: /Users/hal/Documents/workspace/cudl-puppet/augtest.pp, line: 1, column: 1) on node slate.lan

$ AUGEAS_LENS_LIB=/usr/local/share/augeas/lenses/dist puppet apply --modulepath modules augtest.pp
<augeas match="/store">
  <node label="store" path="/store"/>
</augeas>
Error: Evaluation Error: Error while evaluating a Function Call, augeas(): Failed to apply changes with lens Properties.lns:  (file: /Users/hal/Documents/workspace/cudl-puppet/augtest.pp, line: 1, column: 1) on node slate.lan

$ DYLD_LIBRARY_PATH=/usr/local/Cellar/augeas/1.12.0/lib AUGEAS_LENS_LIB=/usr/local/share/augeas/lenses/dist puppet apply --modulepath modules augtest.pp
<augeas match="/store">
  <node label="store" path="/store">
    <node label="msg1">
      <value>abc</value>
    </node>
    <node label="msg2">
      <value>def</value>
    </node>
  </node>
</augeas>
Notice: Compiled catalog for slate.lan in environment production in 0.09 seconds
Notice: Applied catalog in 0.02 seconds

---

The issue seems to be caused by the augeas library — its own build scripts are compiling it incorrectly on OSX: https://github.com/hercules-team/augeas/issues/729

Homebrew has a build of augeas which works around the issue: https://github.com/Homebrew/homebrew-core/issues/83407#issuecomment-901370788

Expected behavior Augeas should work.

Additional context

$ brew info puppetlabs/puppet/pdk
pdk: 2.3.0.0
https://github.com/puppetlabs/pdk
/usr/local/Caskroom/pdk/2.3.0.0 (352MB)
From: https://github.com/puppetlabs/homebrew-puppet/blob/HEAD/Casks/pdk.rb
==> Name
Puppet Development Kit
==> Description
None
==> Artifacts
pdk-2.3.0.0-1-installer.pkg (Pkg)

• pdk version 2.3.0
• macOS 11.6.2

[github-actions[bot]]

This issue has been marked stale because it is open with no recent activity. The PDK Team is actively prioritizing existing bugs and new features, if this issue is still important to you please comment and we will add this to our backlog to complete.

[david22swan]

@h4l Hey, just checking that this is still an issue you are seeing, having some difficulty replicating

[h4l]

Hey @david22swan, thanks for checking. I'm not sure to be honest, as I'm using PDK inside a linux container when developing now, I don't have it installed natively on OSX any longer. This must be quite a niche use-case (combination of using Augeas, and using it via PDK on OSX), so it could be pragmatic to shelve this unless anyone else turns up with the same problem.

[h0tw1r3]

This was fixed by https://github.com/puppetlabs/puppet-runtime/pull/581