Unable to recognize funds post 0.7.2.1 upgrade

mdunnio commented 5 years ago

Issue and Steps to Reproduce

Upgraded to version 0.7.2.1 from version 0.7.1 and funds are no longer being listed via the listfunds command`.

Steps to reproduce:

1) Send funds to lightning wallet on version 0.7.1. Confirm funds. 2) Upgrade to 0.7.2.1 3) run lightning-cli listfunds and see the following:

bitcoin@lightning:/var/lib/bitcoin $ lightning-cli listfunds
{
   "outputs": [],
   "channels": []
}

`getinfo` output

bitcoin@lightning:/var/lib/bitcoin $ lightning-cli getinfo
{
   "id": "020366a9946e4d6b2ad8f9d6d7211fcd37837231d5f9a616296ef02dace1e1070a",
   "alias": "mdunnio",
   "color": "020366",
   "num_peers": 0,
   "num_pending_channels": 0,
   "num_active_channels": 0,
   "num_inactive_channels": 0,
   "address": [],
   "binding": [
      {
         "type": "ipv6",
         "address": "::",
         "port": 9735
      },
      {
         "type": "ipv4",
         "address": "0.0.0.0",
         "port": 9735
      }
   ],
   "version": "v0.7.2.1",
   "blockheight": 592951,
   "network": "bitcoin",
   "msatoshi_fees_collected": 0,
   "fees_collected_msat": "0msat”
}

As a side note: I also noticed that using the lightning-cli with --lightning-dir is producing different results between 0.7.1 and 0.7.2.1. For example:

On version 0.7.1:

bitcoin@lightning:/var/lib/bitcoin $ lightning-cli --lightning-dir=lightning listfunds
{
   "outputs" : [
   ...
}

On version 0.7.2.1:

bitcoin@lightning:/var/lib/bitcoin $ lightning-cli --lightning-dir=lightning listfunds
lightning-cli: Connecting to 'lightning-rpc': Connection refused

To run the command successfully, I have to remove the --lightning-dir param. Not sure if it’s related, but thought I might throw it in here anyways.

Please let me know if I can help debug, or I can grab some log files.

Thanks!

cdecker commented 5 years ago

Thanks for reporting this @mdunnio. I suspect that the two versions (pre- and post-upgrade are actually running in different home directories). This is supported by a downgrade resulting in the funds showing up again, and the fact that after the upgrade the --lightning-dir parameter resulting in the connection being refused. It seems like the old version is being started in the lightningd sub-directory of the current working directory, while the post-upgrade version is using the default $HOME/.lightning/ directory.

Could you share the exact startup parameters you provided with both versions, and the config files in the respective directories?

Likely the most indicative piece of information would be to check the output of lightning-cli getinfo before and after the update. If the node ID doesn't match you are indeed running two different instances.

This might all be caused by a small regression in our handling of the --lightning-dir parameter (we now cd into the directory before touching anything), so this might be the same issue as #3023 and can be temporarily fixed by using absolute paths for --lightning-dir.

mdunnio commented 5 years ago

Looks like you're correct, I am running two different instances. Here is the current output on startup:

Sep 02 20:37:58 lightning lightningd[31201]: 2019-09-02T20:37:58.184Z INFO lightningd(31201): Server started with public key 020366a9946e4d6b2ad8f9d6d7211fcd37837231d5f9a616296ef02dace1e1070a, alias mdunnio (color #020366) and lightningd

vs. the public key that is represented on 1ml.com (0208fbcafddef7e746ff727c2244b854a7f18a52f9ce9a03724a190cc5b999313a)

Here is the command I have systemd running:

/usr/local/bin/lightningd --daemon --conf /var/lib/bitcoin/lightning/lightningd.conf --plugin-dir=/var/lib/bitcoin/lightning/plugins --pid-file=/run/lightningd/lightningd.pid

lightning.conf:

alias=mdunnio
network=bitcoin

bitcoin-rpcuser=****
bitcoin-rpcpassword=****

lightning-dir=/var/lib/bitcoin/lightning

I am unable to retrieve the getinfo output for the previous version right now, but I'll get it as soon as I can. Thanks!

cdecker commented 5 years ago

Hm, ok. I've never seen things done this way around: specify a config-file which then specifies the lightning-dir. Usually it is the other way around (your lightning-dir also dictates where the config file is loaded from).

This is likely something to do with the change in order (cd first then initialize) so we'll need to look into it.

For the time being you can get back up and running by specifying the --lightning-dir option with the absolute path on the command line :wink:

mdunnio commented 5 years ago

Got it to work.

Had to change my command to:

/usr/local/bin/lightningd --daemon --network bitcoin --lightning-dir=/var/lib/bitcoin/lightning --conf /var/lib/bitcoin/lightning/lightningd.conf --plugin-dir=/var/lib/bitcoin/lightning/plugins --pid-file=/run/lightningd/lightningd.pid

It didn’t work if I only specified the --lightning-dir, as it wasn’t able to bring in any of my configuration file variables. I thought that if I passed in the --lightning-dir it would auto-derive the lightningd.conf path?

darosior commented 5 years ago

It didn’t work if I only specified the --lightning-dir, as it wasn’t able to bring in any of my configuration file variables. I thought that if I passed in the --lightning-dir it would auto-derive the lightningd.conf path?

lightningd looks for a configuration file named config, if you want this behavior you should rename your config file to /path/to/lightning_dir/config.

mdunnio commented 5 years ago

@darosior Thanks. I changed the config file to be named config.

cdecker commented 5 years ago

Ok, added #3030 as a new tracking issue for the root cause here. Closing this.

ElementsProject / lightning