search

lightningnetwork / lnd

Lightning Network Daemon ⚡️
MIT License
7.69k stars 2.09k forks source link

LND update & recompiling: frequent problem with updating dependencies #2223

Closed Perlover closed 5 years ago

Perlover commented 5 years ago

Background

If i fetch from github new version of LND and do: make clean && make i often get errors about dependencies

Your environment

Steps to reproduce

I have in $GOPATH/src/github.com/lightningnetwork/lnd the old 0.4.2-beta commit=7cf5ebe2650b6798182e10be198c7ffc1f1d6e19 I do then: 

go get -d github.com/lightningnetwork/lnd &&\
    cd $GOPATH/src/github.com/lightningnetwork/lnd &&\
    git fetch origin && git checkout 42c45979215e3242ef2e162ccd9de3cbb0ed28a1 &&\
    make clean && make && make install

if [ "x`grep '/home/bitcoin/.bitcoin_envs' /home/bitcoin/.bash_profile`" = "x" ]; then echo $'\n. /home/bitcoin/.bitcoin_envs' >> /home/bitcoin/.bash_profile; fi
go get -d github.com/lightningnetwork/lnd &&\
    cd $GOPATH/src/github.com/lightningnetwork/lnd &&\
    git fetch origin && git checkout 42c45979215e3242ef2e162ccd9de3cbb0ed28a1 &&\
    make clean && make && make install
remote: Enumerating objects: 1727, done.
remote: Counting objects: 100% (1727/1727), done.
remote: Compressing objects: 100% (24/24), done.
remote: Total 5910 (delta 1706), reused 1712 (delta 1703), pack-reused 4183
Receiving objects: 100% (5910/5910), 6.20 MiB, done.
Resolving deltas: 100% (3915/3915), completed with 299 local objects.
From https://github.com/lightningnetwork/lnd
   2e838ab..42c4597  master     -> origin/master
 * [new branch]      temp-testnet-fee-estimation -> origin/temp-testnet-fee-estimation
 * [new tag]         v0.5.1-beta-rc4 -> v0.5.1-beta-rc4
From https://github.com/lightningnetwork/lnd
 * [new tag]         v0.5-beta  -> v0.5-beta
 * [new tag]         v0.5-beta-rc1 -> v0.5-beta-rc1
 * [new tag]         v0.5-beta-rc2 -> v0.5-beta-rc2
 * [new tag]         v0.5.1-beta-rc1 -> v0.5.1-beta-rc1
 * [new tag]         v0.5.1-beta-rc2 -> v0.5.1-beta-rc2
 * [new tag]         v0.5.1-beta-rc3 -> v0.5.1-beta-rc3
Previous HEAD position was 7cf5ebe... lnd+cmd/lncli: bump version to 0.4.2
HEAD is now at 42c4597... Merge pull request #1937 from halseth/data-loss-protect-resending
make[1]: Entering directory `/home/bitcoin/go/src/github.com/lightningnetwork/lnd'
\033[0;32m Cleaning source.\033[0m\033[0m
rm -f ./lnd-debug ./lncli-debug
rm -f -r ./vendor .vendor-new
make[1]: Leaving directory `/home/bitcoin/go/src/github.com/lightningnetwork/lnd'
make[1]: Entering directory `/home/bitcoin/go/src/github.com/lightningnetwork/lnd'
\033[0;32m Compiling dependencies.\033[0m
dep ensure -v
Root project is "github.com/lightningnetwork/lnd"
 42 transitively valid internal packages
 66 external packages imported from 30 projects
(0)   ✓ select (root)
(1) ? attempt github.com/NebulousLabs/go-upnp with 1 pkgs; at least 1 versions to try
(1)     try github.com/NebulousLabs/go-upnp@29b680b06c82d044ebea91bf3069038eb562df2a
(1) ✓ select github.com/NebulousLabs/go-upnp@29b680b06c82d044ebea91bf3069038eb562df2a w/7 pkgs
(2) ? attempt github.com/NebulousLabs/fastrand with 1 pkgs; at least 1 versions to try
(2)     try github.com/NebulousLabs/fastrand@master
(2) ✓ select github.com/NebulousLabs/fastrand@master w/1 pkgs
(3) ? attempt github.com/Yawning/aez with 1 pkgs; at least 1 versions to try
(3)     try github.com/Yawning/aez@4dad034d9db2caec23fb8f69b9160ae16f8d46a3
(3) ✓ select github.com/Yawning/aez@4dad034d9db2caec23fb8f69b9160ae16f8d46a3 w/1 pkgs
(4) ? attempt git.schwanenlied.me/yawning/bsaes.git with 2 pkgs; at least 1 versions to try
(4)     try git.schwanenlied.me/yawning/bsaes.git@master
(4) ✗   unable to update checked out version: fatal: reference is not a tree: c0276d75487ef7aceab140c9eb31601dd7d18bd4
(4)   : command failed: [git checkout c0276d75487ef7aceab140c9eb31601dd7d18bd4]: exit status 128
(4)     try git.schwanenlied.me/yawning/bsaes.git@master
(4) ✗   unable to update checked out version: fatal: reference is not a tree: c0276d75487ef7aceab140c9eb31601dd7d18bd4
(4)   : command failed: [git checkout c0276d75487ef7aceab140c9eb31601dd7d18bd4]: exit status 128
(4)   ← no more versions of git.schwanenlied.me/yawning/bsaes.git to try; begin backtrack
(3) ← backtrack: no more versions of github.com/Yawning/aez to try
(3)   ? continue github.com/Yawning/aez with 1 pkgs; 1 more versions to try
(3)     try github.com/Yawning/aez@master
(4) ✗   github.com/Yawning/aez@master not allowed by constraint 4dad034d9db2caec23fb8f69b9160ae16f8d46a3:
(4)     4dad034d9db2caec23fb8f69b9160ae16f8d46a3 from (root)
(3) ← backtrack: no more versions of github.com/Yawning/aez to try
(2) ← backtrack: no more versions of github.com/NebulousLabs/fastrand to try
(1) ← backtrack: no more versions of github.com/NebulousLabs/go-upnp to try
  ✗ solving failed

Solver wall times by segment:
 b-source-exists: 1.960018153s
     b-list-pkgs:  1.74651547s
 b-list-versions:  1.02823401s
      b-gmal:  148.86362ms
     select-root:   6.041454ms
     satisfy:    522.562µs
    new-atom:    355.096µs
     select-atom:    289.881µs
    unselect:    223.128µs
  b-deduce-proj-root:    163.999µs
       backtrack:     94.733µs
       other:      27.76µs
      b-pair-rev:      7.012µs
       b-matches:      3.709µs

  TOTAL: 4.891360587s

Solving failure: No versions of git.schwanenlied.me/yawning/bsaes.git met constraints:
    master: unable to update checked out version: fatal: reference is not a tree: c0276d75487ef7aceab140c9eb31601dd7d18bd4
: command failed: [git checkout c0276d75487ef7aceab140c9eb31601dd7d18bd4]: exit status 128
    master: unable to update checked out version: fatal: reference is not a tree: c0276d75487ef7aceab140c9eb31601dd7d18bd4
: command failed: [git checkout c0276d75487ef7aceab140c9eb31601dd7d18bd4]: exit status 128

I already had same problem with update but your hint to delete vendor dir doesn't work - make clean does always.

I think problem there: $GOPATH/pkg/dep/sources/https---git.schwanenlied.me-yawning-bsaes.git/ I don't know why but go doesn't do git fetch origin and there old repo and there is no fresh commit c0276d75487ef7aceab140c9eb31601dd7d18bd4

How can i automate git fetch for all dependencies of LND and for dependencies of dependencies of dependencies of LND?

Perlover commented 5 years ago

Some news:

I updated go (up to v1.11.2) from sources and after dep (up to v0.5.0 from sources) Then i did:

[bitcoin@HOST lnd]$ pwd
/home/bitcoin/go/src/github.com/lightningnetwork/lnd
[bitcoin@HOST lnd]$ git log|head
commit 42c45979215e3242ef2e162ccd9de3cbb0ed28a1
Merge: 8924d8f b1a35fc
Author: Olaoluwa Osuntokun <laolu32@gmail.com>
Date:   Sun Nov 25 20:47:16 2018 -0600

Merge pull request #1937 from halseth/data-loss-protect-resending

Data loss protect resending

commit 8924d8fb20eb2abfd9cc93c6cc7eb6951184cb88
[bitcoin@HOST lnd]$ dep version
dep:
 version     : v0.5.0
 build date  : 
 git hash    : 
 go version  : go1.11.2
 go compiler : gc
 platform    : linux/amd64
 features    : ImportDuringSolve=false
[bitcoin@HOST lnd]$ go version
go version go1.11.2 linux/amd64
[bitcoin@HOST lnd]$ dep ensure -v
(1/43) Wrote github.com/btcsuite/btclog@84c8d2346e9fc8c7b947e243b9c24e6df9fd206a
(2/43) Wrote github.com/btcsuite/btcutil@ab6388e0c60ae4834a1f57511e20c17b5f78be4b
(3/43) Wrote github.com/jrick/logrotate@a93b200c26cbae3bb09dd0dc2c7c7fe1468a034a
(4/43) Wrote github.com/davecgh/go-spew@8991bc29aa16c548c550c7ff78260e27b9ab7c73
(5/43) Wrote gopkg.in/macaroon.v2@bed2a428da6e56d950bed5b41fcbae3141e5b0d0
(6/43) Wrote github.com/NebulousLabs/fastrand@master
(7/43) Wrote github.com/aead/siphash@83563a290f60225eb120d724600b9690c3fb536f
(8/43) Wrote github.com/btcsuite/go-socks@master
(9/43) Wrote github.com/NebulousLabs/go-upnp@29b680b06c82d044ebea91bf3069038eb562df2a
(10/43) Wrote github.com/btcsuite/websocket@master
(11/43) Wrote github.com/go-errors/errors@a6af135bd4e28680facf08a3d206b454abc877a4
(12/43) Wrote github.com/urfave/cli@1efa31f08b9333f1bd4882d61f9d668a70cd902e
(13/43) Wrote golang.org/x/time@fbb02b2291d28baffd63558aa44b4b56f178d650
(14/43) Wrote github.com/kkdai/bstream@f391b8402d23024e7c0f624b31267a89998fca95
(15/43) Wrote github.com/Yawning/aez@4dad034d9db2caec23fb8f69b9160ae16f8d46a3
(16/43) Wrote github.com/btcsuite/golangcrypto@master
(17/43) Wrote gopkg.in/errgo.v1@b20caedf0710d0988e92b5f2d76843ad1f231f2d
(18/43) Wrote github.com/jackpal/go-nat-pmp@28a68d0c24adce1da43f8df6a57340909ecd7fdd
(19/43) Wrote github.com/btcsuite/btcd@79e00513b1011888b1e675157ab89f527f901cae
(20/43) Wrote gopkg.in/macaroon-bakery.v2@94012773d2874a067572bd16d7d11ae02968b47b
(21/43) Wrote github.com/btcsuite/btcwallet@7ad4f1e81d7831b5b4bc8597fe9db731fbb3be22
(22/43) Wrote github.com/coreos/bbolt@4f5275f4ebbf6fe7cb772de987fa96ee674460a7
(23/43) Wrote github.com/grpc-ecosystem/grpc-gateway@f2862b476edcef83412c7af8687c9cd8e4097c0f
(24/43) Wrote google.golang.org/grpc@8dea3dc473e90c8179e519d91302d0597c0ca1d1
(25/43) Wrote golang.org/x/net@ae89d30ce0c63142b652837da33d782e2b0a9b25
(26/43) Wrote github.com/rogpeppe/fastuuid@master
(27/43) Wrote github.com/lightninglabs/gozmq@master
(28/43) Wrote github.com/miekg/dns@79bfde677fa81ff8d27c4330c35bda075d360641
(29/43) Wrote github.com/jackpal/gateway@v1.0.4
(30/43) Wrote github.com/btcsuite/fastsha256@master
(31/43) Wrote github.com/jessevdk/go-flags@f88afde2fa19a30cf50ba4b05b3d13bc6bae3079
(32/43) Wrote github.com/tv42/zbase32@501572607d0273fc75b3b261fa4904d63f6ffa0e
(33/43) Wrote github.com/golang/protobuf@bbd03ef6da3a115852eaf24c8a1c46aeb39aa175
(34/43) Wrote google.golang.org/genproto@df60624c1e9b9d2973e889c7a1cff73155da81c4
(35/43) Wrote golang.org/x/text@v0.3.0
(36/43) Failed to write golang.org/x/sys@ebe1bf3edb3325c393447059974de898d5133eb8
(37/43) Failed to write github.com/aead/chacha20@master
(38/43) Failed to write git.schwanenlied.me/yawning/bsaes.git@master
(39/43) Failed to write golang.org/x/crypto@614d502a4dac94afa3a6ce146bd1736da82514c6
(40/43) Failed to write github.com/ltcsuite/ltcd@cdab10132e8c6e4a3ffd112dba54791946d28906
(41/43) Failed to write github.com/lightninglabs/neutrino@9a42f7df21be82a69f04caa83bce4034dca72764
(42/43) Failed to write github.com/lightningnetwork/lightning-onion@ac4d9da8f1d67c95f1fafdc65e1a4902d6f5a940
(43/43) Failed to write github.com/juju/loggo@master
grouped write of manifest, lock and vendor: error while writing out vendor tree: failed to write dep tree: failed to export golang.org/x/sys: fatal: failed to unpack tree object ebe1bf3edb3325c393447059974de898d5133eb8
: exit status 128
[bitcoin@HOST lnd]$ dep ensure -v -update
Root project is "github.com/lightningnetwork/lnd"
 42 transitively valid internal packages
 66 external packages imported from 30 projects
(0)   ✓ select (root)
(1) ? attempt github.com/NebulousLabs/go-upnp with 1 pkgs; 5 versions to try
(1)     try github.com/NebulousLabs/go-upnp@29b680b06c82d044ebea91bf3069038eb562df2a
(1) ✓ select github.com/NebulousLabs/go-upnp@29b680b06c82d044ebea91bf3069038eb562df2a w/7 pkgs
(2) ? attempt github.com/NebulousLabs/fastrand with 1 pkgs; 8 versions to try
(2)     try github.com/NebulousLabs/fastrand@master
(2) ✓ select github.com/NebulousLabs/fastrand@master w/1 pkgs
(3) ? attempt github.com/btcsuite/btcwallet with 8 pkgs; 12 versions to try
(3)     try github.com/btcsuite/btcwallet@7ad4f1e81d7831b5b4bc8597fe9db731fbb3be22
(3) ✓ select github.com/btcsuite/btcwallet@7ad4f1e81d7831b5b4bc8597fe9db731fbb3be22 w/16 pkgs
(4) ? attempt github.com/Yawning/aez with 1 pkgs; 2 versions to try
(4)     try github.com/Yawning/aez@4dad034d9db2caec23fb8f69b9160ae16f8d46a3
(4) ✓ select github.com/Yawning/aez@4dad034d9db2caec23fb8f69b9160ae16f8d46a3 w/1 pkgs
(5) ? attempt git.schwanenlied.me/yawning/bsaes.git with 2 pkgs; 1 versions to try
(5)     try git.schwanenlied.me/yawning/bsaes.git@master
(5) ✗   unable to update checked out version: fatal: reference is not a tree: c0276d75487ef7aceab140c9eb31601dd7d18bd4
(5)   : command failed: [git checkout c0276d75487ef7aceab140c9eb31601dd7d18bd4]: exit status 128
(5)   ← no more versions of git.schwanenlied.me/yawning/bsaes.git to try; begin backtrack
(4) ← backtrack: no more versions of github.com/Yawning/aez to try
(4)   ? continue github.com/Yawning/aez with 1 pkgs; 1 more versions to try
(4)     try github.com/Yawning/aez@master
(5) ✗   github.com/Yawning/aez@master not allowed by constraint 4dad034d9db2caec23fb8f69b9160ae16f8d46a3:
(5)     4dad034d9db2caec23fb8f69b9160ae16f8d46a3 from (root)
(4) ← backtrack: no more versions of github.com/Yawning/aez to try
(3) ← backtrack: no more versions of github.com/btcsuite/btcwallet to try
(2) ← backtrack: no more versions of github.com/NebulousLabs/fastrand to try
(1) ← backtrack: no more versions of github.com/NebulousLabs/go-upnp to try
  ✗ solving failed

Solver wall times by segment:
 b-list-versions: 2.207507827s
 b-source-exists: 617.553607ms
      b-gmal: 197.312067ms
     select-root:   5.841322ms
     b-list-pkgs:   1.996249ms
     satisfy:   1.632637ms
    new-atom:   1.381474ms
     select-atom:   1.217972ms
    unselect:    882.251µs
  b-deduce-proj-root:     209.35µs
       backtrack:     64.276µs
       other:     28.453µs

  TOTAL: 3.035627485s

Solving failure: No versions of git.schwanenlied.me/yawning/bsaes.git met constraints:
    master: unable to update checked out version: fatal: reference is not a tree: c0276d75487ef7aceab140c9eb31601dd7d18bd4
: command failed: [git checkout c0276d75487ef7aceab140c9eb31601dd7d18bd4]: exit status 128
Perlover commented 5 years ago

I resolved this problem by this: rm -rf $GOPATH/pkg/dep and then make again from lnd directory. But this is not true resolving. I think if golang uses dependencies and managment of dependecies it should resolved by other way - by dep utility... May be lnd has some incorrect file Gopkg.toml?

Roasbeef commented 5 years ago

This is a dep issue, we'll be switching over to use the new go modules feature pretty soon here. I'm also unable to reproduce on my end.

ecurrencyhodler commented 5 years ago

I also ran into this issue also. Just leaving a comment for anyone else who who does. It's because I was on a VPN.