Unable to add or pay invoices

[jimbojw]

I'm using LndHub with BTCPayServer (docker). When trying to create an invoice in BlueWallet, after a while the app shows an Alert dialog indicating that the call to /addinvoice had timed out, showing an HTTP 504 error from nginx:

API failure: Error {
  "type": "default",
  "status": 504,
  "ok": false,
  "statusText": "",
  "headers": {..., "server": "nginx",...},
  "url": "https://<myservername>/bluewallet_lndhub_app/addinvoice",
  ...
} 

In the LndHub logs, I can see the beginning of the /addinvoice router processing:

$ docker logs --tail 1000 bluewallet_lndhub_app
updateLightning()
updateLightning()
updateDescribeGraph()
2022-08-02T18:00:48.528Z : info: [/addinvoice] : ["<my-req-id>"]
2022-08-02T18:00:48.533Z : info: [/addinvoice] : ["<my-req-id>","userid: <myuserid>"]

So I know it made it to at least line 185: https://github.com/BlueWallet/LndHub/blob/master/controllers/api.js#L185

After a long while, the logs show the POST. My guess is that this is when nginx broke off waiting, but I'm not sure.

updateLightning()
<my-req-id> <my ip address> - - [02/Aug/2022:18:01:48 +0000] "POST /addinvoice HTTP/1.1" - - "-" "BlueWallet/1656866507 CFNetwork/1333.0.4 Darwin/21.5.0"
updateLightning()
updateDescribeGraph()

I also cannot pay invoices. Paying gives less information. The Alert box just says "Network request failed". Logs are similar.

I suspect that there's some kind of connection issue with LndHub talking to lnd but I don't know what to do next to investigate further. This used to work, so I'm wondering if a certificate or network configuration changed in some way. I am able to create new accounts, so I don't think it's a problem with Redis.

[Overtorment]

LND's certificates expire once a year I think. have you checked those?

[jimbojw]

I will check. I thought I recently updated them, but I could be mistaken.

Is there any chance that LndHUB is using the wrong one? How would I check this?

Is there any way for LndHUB to report that kind of error explicitly?

[dennisreimann]

Is there any chance that LndHUB is using the wrong one?

The BTCPay Server LNDhub fragment mounts the LND data directory and uses the tls.cert from there.

For debugging, something like this should work:

docker exec -i generated_bluewallet_lndhub_app_1 cat /lnd/tls.cert | openssl x509 -text -noout

Maybe you need to adapt the container name, you can find out the name using docker ps.

[jimbojw]

The BTCPay Server LNDhub fragment mounts the LND data directory and uses the tls.cert from there.

Thanks, I saw that when I was poking around on the command line. It looked to me like it was using the canonical cert and macaroon, but I wasn't sure.

For debugging, something like this should work:

docker exec -i generated_bluewallet_lndhub_app_1 cat /lnd/tls.cert | openssl x509 -text -noout

Maybe you need to adapt the container name, you can find out the name using docker ps.

I don't seem to have a generated_bluewallet_lndhub_app_1 instance, but using docker ps I was able to find a bluewallet_lndhub_app instance:

Results (some info removed):

$ docker exec -i bluewallet_lndhub_app cat /lnd/tls.cert | openssl x509 -text -noout
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            ...
    Signature Algorithm: ecdsa-with-SHA256
        Issuer: O=lnd autogenerated cert, CN=...
        Validity
            Not Before: Jul 18 19:42:18 2022 GMT
            Not After : Sep 12 19:42:18 2023 GMT
        Subject: O=lnd autogenerated cert, CN=...
        Subject Public Key Info:
            Public Key Algorithm: id-ecPublicKey
                Public-Key: (256 bit)
                pub: 
                    ...
                ASN1 OID: prime256v1
                NIST CURVE: P-256
    ...

So it looks to me like LndHUB is indeed using the cert which I created recently. July 18 must have been when I updated it, which sounds about right to my knowledge. I'm not sure exactly when I stopped being able to make and pay Blue Wallet invoices because I didn't notice until a few weeks later.

Could it be a macaroon issue? What else should I try?

Interestingly enough, the LndHUB info page linked from BTCPayServer services DOES show my open channels. So it's able to get some data, just not making/paying invoices apparently.

[jimbojw]

Also, FWIW, other clients of lnd continue to be able to make/pay invoices, such as RTL and Zeus.

[jimbojw]

UPDATE: Apparently Blue Wallet IS creating invoices through LndHUB. I see them appear in RTL and in Zeus. It's just that they never show up in Blue Wallet (Status 504 API Errors as described in the original issue report).

[dennisreimann]

Can you try btcpay-restart.sh? This will allow LND and LNDhub to pick up the changes tls.cert, which maybe didn't happen. Depends on how you renewed it.

[jimbojw]

Thanks! That seems to have fixed it. The test invoices I created from Blue Wallet disappeared after restart, but I am now again able to create invoices.