Cannot run Aperture: unable to parse config file: invalid authenticator configuration

[secondl1ght]

Hello, I am trying to get Aperture running on a server and I am receiving the following error when I run the aperture command after building:

unable to parse config file: invalid authenticator configuration

You can view the config I am using here and the diff from the defaults: https://github.com/lightninglabs/aperture/compare/master...secondl1ght:aperture:master

I have the auth disabled so I am not sure why it is not liking this config. All the changes made to the config were from following the instructions about setting up Aperture + a LNC Mailbox in the Builder's Guide (https://docs.lightning.engineering/lightning-network-tools/aperture).

Any help to get this working would be appreciated. Thanks!

[hieblmi]

Are you able to post your configuration here?

My authenticator lines look something like this:

 12 authenticator:                                                                  
 13   lndhost: localhost:10013                                                      
 14   tlspath: .../docker/mounts/regtest/charlie/tls.cert                    
 15   macdir: ...docker/mounts/regtest/charlie/                             
 16   network: ...    

[secondl1ght]

# The address which the proxy can be reached at.
listenaddr: "lnc-mailbox.onrender.com:443"

# The root path of static content to serve upon receiving a request the proxy
# cannot handle.
staticroot: "./static"

# Should the static file server be enabled that serves files from the directory
# specified in `staticroot`?
servestatic: false

# The log level that should be used for the proxy.
#
# Valid options include: trace, debug, info, warn, error, critical, off.
debuglevel: "trace"

# Whether the proxy should create a valid certificate through Let's Encrypt for
# the fully qualifying domain name.
autocert: true
servername: lnc-mailbox.onrender.com

# The port on which the pprof profile will be served. If no port is provided,
# the profile will not be served.
profile: 9999

# Settings for the lnd node used to generate payment requests. All of these
# options are required.
authenticator:
  # The host:port which lnd's RPC can be reached at.
  lndhost: "localhost:10009"

  # The path to lnd's TLS certificate.
  tlspath: "/path/to/lnd/tls.cert"

  # The path to lnd's macaroon directory.
  macdir: "/path/to/lnd/data/chain/bitcoin/simnet"

  # The chain network the lnd is active on.
  network: "simnet"

  # The LNC connection passphrase.
  passphrase: "my-own-passphrase"

  # The host:port of the mailbox server to be used.
  mailboxaddress: "mailbox.terminal.lightning.today:443"

  # Set to true to skip verification of the mailbox server's tls cert.
  devserver: false

  # Set to true to disable any auth.
  disable: true

# The selected database backend. The current default backend is "sqlite". 
# Aperture also has support for postgres and etcd.
dbbackend: "sqlite"

# Settings for the sqlite process which the proxy will use to reliably store and
# retrieve token information.
sqlite:
    # The full path to the database.
    dbfile: "/path/to/.aperture/aperture.db"

# Settings for the postgres instance which the proxy will use to reliably store 
# and retrieve token information.
postgres:
    # Connection parameters.
    host: "localhost"
    port: 5432
    user: "user"
    password: "password"
    dbname: "aperture"

    # Max open connections to keep alive to the database server.
    maxconnections: 25

    # Whether to require using SSL (mode: require) when connecting to the 
    # server.
    requireSSL: true

# Settings for the etcd instance which the proxy will use to reliably store and
# retrieve token information.
etcd:
  # The client host:port which the etcd instance can be reached at.
  host: "localhost:2379"

  # If authentication is enabled, the user and password required to access the
  # etcd instance.
  user: "user"
  password: "password"

# List of services that should be reachable behind the proxy.  Requests will be
# matched to the services in order, picking the first that satisfies hostregexp
# and (if set) pathregexp. So order is important!
#
# Use single quotes for regular expressions with special characters in them to
# avoid YAML parsing errors!
services:
    # The identifying name of the service. This will also be used to identify
    # which capabilities caveat (if any) corresponds to the service.
  - name: "service1"

    # The regular expression used to match the service host.
    hostregexp: '^service1.com$'

    # The regular expression used to match the path of the URL.
    pathregexp: '^/.*$'

    # The host:port which the service can be reached at.
    address: "127.0.0.1:10009"

    # The HTTP protocol that should be used to connect to the service. Valid
    # options include: http, https.
    protocol: https

    # If required, a path to the service's TLS certificate to successfully
    # establish a secure connection.
    tlscertpath: "path-to-optional-tls-cert/tls.cert"

    # A comma-delimited list of capabilities that will be granted for tokens of
    # the service at the base tier.
    capabilities: "add,subtract"

    # The set of constraints that are applied to tokens of the service at the
    # base tier.
    constraints:
        # This is just an example of how aperture could be extended
        # but would not have any effect without additional support added.
        "valid_until": 1682483169

    # a caveat will be added that expires the LSAT after this many seconds,
    # 31557600 = 1 year.
    timeout: 31557600    

    # The LSAT value in satoshis for the service. It is ignored if
    # dynamicprice.enabled is set to true.
    price: 0

    # Options to use for connection to the price serving gRPC server.
    dynamicprice:
      # Whether or not a gRPC server is available to query price data from. If
      # this option is set to true then the 'price' option is ignored.
      enabled: true

      # The address of the gRPC pricer server.
      grpcaddress: "127.0.0.1:10010"

      # Whether or not TLS encryption should be used for communications with the
      # gRPC server.
      insecure: false

      # The path to the pricer server's tls.cert. If the 'insecure' option is
      # set to true then this path must be set.
      tlscertpath: "path-to-pricer-server-tls-cert/tls.cert"

  - name: "service2"
    hostregexp: "service2.com:8083"
    pathregexp: '^/.*$'
    address: "123.456.789:8082"
    protocol: https
    constraints:
        "valid_until": "2020-01-01"
    price: 1

  - name: "service3"
    hostregexp: "service3.com:8083"
    pathregexp: '^/.*$'
    address: "123.456.789:8082"
    protocol: https
    constraints:
        "valid_until": "2020-01-01"
    dynamicprice:
      enbled: true
      grpcaddress: 123.456.789:8083
      insecure: false
      tlscertpath: "path-to-pricer-server-tls-cert/tls.cert"

# Settings for a Tor instance to allow requests over Tor as onion services.
# Configuring Tor is optional.
tor:
  # The host:port which Tor's control can be reached at.
  control: "localhost:9051"

  # The internal port we should listen on for client requests over Tor. Note
  # that this port should not be exposed to the outside world, it is only
  # intended to be reached by clients through the onion service.
  listenport: 8082

  # The port through which the onion services to be created can be reached at.
  virtualport: 8082

  # Whether a v3 onion service should be created to handle requests.
  v3: false

# Enable the Lightning Node Connect hashmail server, allowing up to 1k messages
# per burst and a new message every 20 milliseconds.
hashmail:
  enabled: true
  messagerate: 1ms
  messageburstallowance: 99999999

# Enable the prometheus metrics exporter so that a prometheus server can scrape
# the metrics.
prometheus:
  enabled: false
  listenaddr: "localhost:9000"

[hieblmi]

Hmm, I don't run into your error. You could try and start with a minimal configuration that works and then extend the parameters so you narrow down which parts of the config are causing the issue?

[secondl1ght]

I wonder why you don't get the error with the same config... do you know what the minimum config required for a LNC Mailbox is?

I am just following the docs here: https://docs.lightning.engineering/lightning-network-tools/aperture/mailbox#docs-internal-guid-b757d186-7fff-3163-6ef9-f86657a3772a so that's the config that I am using. But I am not sure what is needed, I used the sample config and then changed what was stated in the docs.

Also are you running locally during testing or with a server provider? Currently I am testing with Render (https://render.com/) but is there a different host you recommend to use that you have had success in deploying Aperture with?

[hieblmi]

I am running locally.

I tagged you in a conversation on Slack where a user achieved the same setup that you are trying to set up. You might be able to get some info from them: https://lightningcommunity.slack.com/archives/C0112B0JYP2/p1696708659029809?thread_ts=1695837617.272239&cid=C0112B0JYP2.

[secondl1ght]

Thanks I checked out that thread and responded - doesn't appear to be the same solution for me though. For reference here is the build command I run:

make install && mkdir ~/.aperture && cp sample-conf.yaml ~/.aperture/aperture.yaml

[guggero]

Since you have disable: true for the authenticator, you should never get that error message. So my suspicion is that the config file as a whole isn't parsed correctly. Could very well be that our sample config has something weird in it that causes values to be ignored. Can you try starting from an empty config file just adding what you need? Or run the config file through a YAML parser and see if all values actually get picked up? Because YAML is susceptible to incorrect white space characters, maybe something isn't properly indented or something like that.

[secondl1ght]

This sounded promising but unfortunately didn't fix the issue. I agree that the error message doesn't make sense since I have auth disabled. If the error message was more specific we could debug easier. Here is the formatted yaml I used in the second attempt for record:

listenaddr: 'lnc-mailbox.onrender.com:443'
staticroot: ./static
servestatic: false
debuglevel: trace
autocert: true
servername: lnc-mailbox.onrender.com
profile: 9999
authenticator:
  lndhost: 'localhost:10009'
  tlspath: /path/to/lnd/tls.cert
  macdir: /path/to/lnd/data/chain/bitcoin/simnet
  network: simnet
  passphrase: my-own-passphrase
  mailboxaddress: 'mailbox.terminal.lightning.today:443'
  devserver: false
  disable: true
dbbackend: sqlite
sqlite:
  dbfile: /path/to/.aperture/aperture.db
postgres:
  host: localhost
  port: 5432
  user: user
  password: password
  dbname: aperture
  maxconnections: 25
  requireSSL: true
etcd:
  host: 'localhost:2379'
  user: user
  password: password
services:
  - name: service1
    hostregexp: ^service1.com$
    pathregexp: ^/.*$
    address: '127.0.0.1:10009'
    protocol: https
    tlscertpath: path-to-optional-tls-cert/tls.cert
    capabilities: 'add,subtract'
    constraints:
      valid_until: 1682483169
    timeout: 31557600
    price: 0
    dynamicprice:
      enabled: true
      grpcaddress: '127.0.0.1:10010'
      insecure: false
      tlscertpath: path-to-pricer-server-tls-cert/tls.cert
  - name: service2
    hostregexp: 'service2.com:8083'
    pathregexp: ^/.*$
    address: '123.456.789:8082'
    protocol: https
    constraints:
      valid_until: '2020-01-01'
    price: 1
  - name: service3
    hostregexp: 'service3.com:8083'
    pathregexp: ^/.*$
    address: '123.456.789:8082'
    protocol: https
    constraints:
      valid_until: '2020-01-01'
    dynamicprice:
      enbled: true
      grpcaddress: '123.456.789:8083'
      insecure: false
      tlscertpath: path-to-pricer-server-tls-cert/tls.cert
tor:
  control: 'localhost:9051'
  listenport: 8082
  virtualport: 8082
  v3: false
hashmail:
  enabled: true
  messagerate: 1ms
  messageburstallowance: 99999999
prometheus:
  enabled: false
  listenaddr: 'localhost:9000'

[guggero]

Can you try removing everything from the authenticator section except the disable? And be careful with un-escaped regular expressions in the services section. Maybe remove that section completely just to get to the bottom of things?

[secondl1ght]

This is the yaml formatter I used for reference: https://jsonformatter.org/yaml-formatter

Here is the new config I am using after removing those items:

listenaddr: 'lnc-mailbox.onrender.com:443'
staticroot: ./static
servestatic: false
debuglevel: trace
autocert: true
servername: lnc-mailbox.onrender.com
profile: 9999
authenticator:
  disable: true
dbbackend: sqlite
sqlite:
  dbfile: /path/to/.aperture/aperture.db
postgres:
  host: localhost
  port: 5432
  user: user
  password: password
  dbname: aperture
  maxconnections: 25
  requireSSL: true
etcd:
  host: 'localhost:2379'
  user: user
  password: password
tor:
  control: 'localhost:9051'
  listenport: 8082
  virtualport: 8082
  v3: false
hashmail:
  enabled: true
  messagerate: 1ms
  messageburstallowance: 99999999
prometheus:
  enabled: false
  listenaddr: 'localhost:9000'

(Still getting same error for some reason...)

[hieblmi]

The parsing of this file works for me. Could you specify that config file when starting aperture, like aperture --configfile=.... Maybe aperture isn't picking up the right file?

[secondl1ght]

OK I am getting a new error now which is good so I will follow this one and report back, thanks!

[secondl1ght]

So the new error I am getting now that the config file is working is: [ERR] APER: Error while running aperture: listen tcp 216.24.57.3:443: bind: cannot assign requested address

[guggero]

That probably either means your port 443 is already occupied. Or you need to run as root to listen on ports < 1024.

[secondl1ght]

Thanks for the help! I think this might be my last error: http: TLS handshake error from 10.213.23.159:46334: EOF Any idea about this one?

[guggero]

That looks like a client connecting but not finishing the TLS handshake. If you have the port open to the internet, then that's probably just a scanning software, you can very likely ignore the error.

[secondl1ght]

Ok thanks, my Aperture server is now running with the config I mentioned above. I am trying to test that the LNC Mailbox is working correctly by connecting to Lightning Terminal (https://terminal.lightning.engineering/) with a node and using the additional options to specify the domain and port of my LNC Mailbox that I just configured. However it is not connecting and throwing an error when trying to connect. Is there a command I can run on the server to make sure the LNC Mailbox is setup correctly and accessible for connections?

[guggero]

Are you also specifying the custom domain and port when attempting the connection on terminal.lightning.engineering? Do you have a valid TLS certificate? If not, on the client (litd) side you might need to specify the devServer flag (can probably only be done in the CLI) and in the browser you'll need to add an exception for the certificate. If that's not the issue, make sure to check the browser console for logs (F12)

[secondl1ght]

Yes I am adding the custom domain and port (lnc-mailbox.onrender.com:8081) on the public instance of Terminal, that is the only place I need to enter it right? And yes the server Aperture is running on does use HTTPS (this is the address https://lnc-mailbox.onrender.com/). The errors in the console when the attempted connection fails on the Terminal website and toast message are:

FAILED TO CONNECT
Not Connected. Failed to connect the WASM client to the proxy server. Try reloading the page or obtaining a new pairing phrase.

Also it might be worth noting my setup for testing this is a Voltage node.

[guggero]

Do you have another proxy inbetween somewhere? When I open https://lnc-mailbox.onrender.com/ in the browser, I immediately get the error Client sent an HTTP request to an HTTPS server.. But since my browser definitely sent a HTTPS request, there's another component in the loop that's misconfigured.

[secondl1ght]

I noticed that as well, I don't know why that would be the case but I will try looking at the Render docs to see if there is anything about how they configure their servers and report back...

[secondl1ght]

Sounds like Render does have a proxy that downgrades the connection to HTTP before it reaches the server. So I will have to use a different server hosting provider in order to get this working. I will make another attempt and I think it should work this time hopefully!

[secondl1ght]

I am looking into Digital Ocean and it seems like they also use HTTP for the internal request on the server to the service that is running. Would it be possible to make Aperture work without HTTPS in the config file? It seems like using HTTPS for the public request and then HTTP for the private proxy request on the server is a common pattern for these hosting providers. My goal is to have a public LNC Mailbox that anyone could access so it needs to have no additional config for the user like the Lightning Labs flagship instance. Would de-bundling the LNC Mailbox from Aperture help with making it easier to deploy an instance? I'm not sure how it all works but maybe the LNC Mailbox could be its own standalone repo. It seems like getting Aperture running locally is pretty straightforward but trying to deploy a production version is a bit harder. Although I am not very familiar with the backend so maybe it's not that hard and I just don't know the steps. I brought up in the Voltage Discord a suggestion for them to have one-click Aperture servers which would be cool. Are you able to share how mailbox.terminal.lightning.today:443 is hosted/configured and served so others could use the same approach?

[guggero]

There is an insecure config option that's missing in the example config file that disables TLS.

[secondl1ght]

Ah ok cool, just to confirm this would still be secure in this particular use-case? Because the main traffic from the client is encrypted using HTTPS to the mailbox server but then internally the server uses HTTP to access the Aperture instance. So it is not actually insecure?

[secondl1ght]

I think I will still try and configure a public LNC Mailbox that uses HTTPS for everything so I don't need to use that insecure flag, I am guessing the Lightning Labs instance is setup this way. From some preliminary research it sounds like I would need to get the GO server setup to receive traffic directly rather than being forwarded traffic on a private network from something like nginx that is the actual public server. I'd also like to figure out what the minimum config is for a LNC Mailbox on Aperture to keep it as simple as possible. Maybe the docs could be updated to provide a bit more information about the best way to deploy the software. But I will close this issue as it is resolved and open a new one if needed. Thanks again for the help and answering my questions about this.