Make path to Augeas configurable

[bmw]

As reported to Homebrew and Certbot, python-augeas may be unusable on macOS when Apple's System Integrity Protection (SIP) is enabled. This is because augeas might not be installed in the default paths on macOS and environment variables for the linker are cleared by SIP.

Currently the only workaround the Homebrew maintainers are aware of are to manually edit cffi.py. Unless there's another workaround we're not aware of, can another solution be added? One solution could be allowing the package maintainer/user to set an environment variable with the path to augeas and fall back to the current string if the environment variable is not set.

[lutter]

@bmw not sure why I didn't see this earlier ... I have no objection to this approach, but I fear my knowledge of how these things work with Python FFI might not be enough to pull this off. Is there a sample/patch that I could look at ?

[lutter]

@thedrow since you rewrote these bindings, can you have a look at this ?

[thedrow]

Sure I'll have a look

[thedrow]

@bmw It does look like you can use the standard LD_LIBRARY_PATH environment when installing/running augeas. I don't have access to a Mac OS to confirm though.

[bmw]

My understanding is LD_LIBRARY_PATH is unset when SIP is enabled. @alyssais, are you able to help with this?

[alyssais]

@bmw My understanding is that SIP only prevents using LD_LIBRARY_PATH on binaries in system locations. So if we're using system Python, it won't work.

[alyssais]

Actually, no, LD_LIBRARY_PATH should be fine.

Any dynamic linker (dyld) environment variables, such as DYLD_LIBRARY_PATH, are purged when launching protected processes.

It's DYLD_LIBRARY_PATH that you can't set.

[thedrow]

So you can point to libaugeas but since we've reverted to direct FFI the module is loaded dynamically right? Hmm, maybe we can provide both the static compilation which requires libxml2 and the dynamically loaded FFI approach (which is slower but has less dependencies).

[oliveratgithub]

I have SIP disabled on my macOS High Sierra 10.13.3 and still get this error when trying to install certbot:

OSError: ctypes.util.find_library() did not manage to locate a library called 'augeas'

$ sudo csrutil status
System Integrity Protection status: disabled.

[basak]

When I was last looking at this problem, I think I found that LD_LIBRARY_PATH isn't used when you provide the name as just augeas. In any case setting LD_LIBRARY_PATH didn't work for me in the certbot snap. I think if you specify a filename eg. libaugeas.so.0 then the behaviour may be different.

This will need testing - I'm not actively working on this right now.

[laffer1]

I realize this is specific to mac os here, but just wanted to point out I had a similar problem on MidnightBSD. Changing the dlopen line to be the name of the library seemed to fix it. Setting the library path will let you build it, but when you go to use it, you must always set it as well.

`--- augeas/ffi.py.orig 2020-03-01 17:24:34.176504000 -0500 +++ augeas/ffi.py 2020-03-01 17:24:54.253657000 -0500 @@ -39,7 +39,7 @@ void free(void *); """)

-lib = ffi.dlopen("augeas") +lib = ffi.dlopen("libaugeas.so")

if name == "main": ffi.compile(verbose=True)`

[bmw]

Homebrew's issue with SIP is resolved because they are no longer using the version of Python provided by Apple for Certbot which was causing environment variables to be unset. See https://github.com/Homebrew/homebrew-core/issues/16757#issuecomment-351379160. If you still hit a problem that looks like this while using Homebrew, you should report the problem with Homebrew.

The other path issue reported here about LD_LIBRARY_PATH doesn't seem to me to be due to anything specific in python-augeas and are instead limitations in how ctypes (which cffi uses internally) works. See https://docs.python.org/3/library/ctypes.html#finding-shared-libraries about the tools ctypes.util.find_library uses to find shared libraries on Linux.

Because of all this, I think this issue can be closed.