libp2p / go-libp2p

libp2p implementation in Go
MIT License
6.05k stars 1.07k forks source link

Unable to dial peers #1007

Closed Strernd closed 4 years ago

Strernd commented 4 years ago

I'm using libp2p to communicate with other instances of the service. All of them are hosted in a docker container on an AWS EC2 machine. They're all configured to each listen on port 5500. The port 5500 is open to all instances of the service. I used the code mentioned in https://github.com/libp2p/go-libp2p/issues/460 .

    extMultiAddr, err := ma.NewMultiaddr(fmt.Sprintf("/ip4/%s/tcp/%d", ipStr, port))
    if err != nil {
        log.Fatalf("Error creating multiaddress: %v\n", err)
    }
    addressFactory := func(addrs []ma.Multiaddr) []ma.Multiaddr {
        if extMultiAddr != nil {
            addrs = append(addrs, extMultiAddr)
        }
        return addrs
    }

    opts := []libp2p.Option{
        libp2p.AddrsFactory(addressFactory),
        libp2p.Identity(privateKey),
    }

It works locally in a docker-compose setup.

The nodes are announcing their address, port and Id over another service and then try to connect to their peers. I tried it with the public and private address of the EC2 machine, but had no success.

failed to dial QmdpHd43HwABMrctDyKkhwBBK2vX963mbdYsNAnrMpT8SD: all dials failed
ss_1         |   * [/ip4/172.31.37.27/tcp/5500] dial tcp4 xxx.xx.xxx.xx:5500: connect: connection refused

I'm not sure if this is a bug, but I'd expect it to work, as I'm doing it in a similar way (at least for the network configuration) with a different service in libp2p-js.

Version Information
github.com/tixl/secret-sharing-test
bou.ke/monkey v1.0.1
cloud.google.com/go v0.26.0
github.com/AndreasBriese/bbloom v0.0.0-20190306092124-e2d15f34fcf9
github.com/BurntSushi/toml v0.3.1
github.com/Kubuxu/go-os-helper v0.0.1
github.com/OneOfOne/xxhash v1.2.2
github.com/aead/siphash v1.0.1
github.com/agl/ed25519 v0.0.0-20170116200512-5312a6153412
github.com/apex/log v1.9.0
github.com/apex/logs v1.0.0
github.com/aphistic/golf v0.0.0-20180712155816-02c07f170c5a
github.com/aphistic/sweet v0.2.0
github.com/armon/consul-api v0.0.0-20180202201655-eb2c6b5be1b6
github.com/aws/aws-sdk-go v1.20.6
github.com/aybabtme/rgbterm v0.0.0-20170906152045-cc83f3b3ce59
github.com/benbjohnson/clock v1.0.3
github.com/binance-chain/tss-lib v1.3.2
github.com/btcsuite/btcd v0.20.1-beta
github.com/btcsuite/btclog v0.0.0-20170628155309-84c8d2346e9f
github.com/btcsuite/btcutil v1.0.2
github.com/btcsuite/go-socks v0.0.0-20170105172521-4720035b7bfd
github.com/btcsuite/goleveldb v0.0.0-20160330041536-7834afc9e8cd
github.com/btcsuite/snappy-go v0.0.0-20151229074030-0bdef8d06723
github.com/btcsuite/websocket v0.0.0-20150119174127-31079b680792
github.com/btcsuite/winsvc v1.0.0
github.com/census-instrumentation/opencensus-proto v0.2.1
github.com/cespare/xxhash v1.1.0
github.com/client9/misspell v0.3.4
github.com/coreos/etcd v3.3.10+incompatible
github.com/coreos/go-etcd v2.0.0+incompatible
github.com/coreos/go-semver v0.3.0
github.com/cpuguy83/go-md2man v1.0.10
github.com/ctrlrsf/logdna v0.0.0-20160923145940-e8165275b7a7
github.com/davecgh/go-spew v1.1.1
github.com/davidlazar/go-crypto v0.0.0-20170701192655-dcfb0a7ac018
github.com/decred/dcrd/dcrec/edwards/v2 v2.0.0
github.com/dgraph-io/badger v1.6.1
github.com/dgraph-io/ristretto v0.0.2
github.com/dgryski/go-farm v0.0.0-20190423205320-6a90982ecee2
github.com/dustin/go-humanize v1.0.0
github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473
github.com/envoyproxy/protoc-gen-validate v0.1.0
github.com/evalphobia/go-logdna v0.1.0
github.com/evalphobia/httpwrapper v0.2.1
github.com/fatih/color v1.7.0
github.com/flynn/noise v0.0.0-20180327030543-2492fe189ae6
github.com/fsnotify/fsnotify v1.4.7
github.com/gin-contrib/sse v0.1.0
github.com/gin-gonic/gin v1.6.3
github.com/go-check/check v0.0.0-20180628173108-788fd7840127
github.com/go-logfmt/logfmt v0.4.0
github.com/go-playground/assert/v2 v2.0.1
github.com/go-playground/locales v0.13.0
github.com/go-playground/universal-translator v0.17.0
github.com/go-playground/validator/v10 v10.2.0
github.com/gogo/protobuf v1.3.1
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6
github.com/golang/mock v1.1.1
github.com/golang/protobuf v1.4.2
github.com/golang/snappy v0.0.0-20180518054509-2e65f85255db
github.com/google/go-cmp v0.5.0
github.com/google/gofuzz v1.0.0
github.com/google/gopacket v1.1.17
github.com/google/renameio v0.1.0
github.com/google/uuid v1.1.1
github.com/gorilla/websocket v1.4.2
github.com/gxed/hashland/keccakpg v0.0.1
github.com/gxed/hashland/murmur3 v0.0.1
github.com/hashicorp/errwrap v1.0.0
github.com/hashicorp/go-multierror v1.1.0
github.com/hashicorp/golang-lru v0.5.4
github.com/hashicorp/hcl v1.0.0
github.com/hpcloud/tail v1.0.0
github.com/huin/goupnp v1.0.0
github.com/huin/goutil v0.0.0-20170803182201-1ca381bf3150
github.com/inconshreveable/mousetrap v1.0.0
github.com/ipfs/go-cid v0.0.7
github.com/ipfs/go-datastore v0.4.4
github.com/ipfs/go-detect-race v0.0.1
github.com/ipfs/go-ds-badger v0.2.3
github.com/ipfs/go-ds-leveldb v0.4.2
github.com/ipfs/go-ipfs-delay v0.0.0-20181109222059-70721b86a9a8
github.com/ipfs/go-ipfs-util v0.0.2
github.com/ipfs/go-log v1.0.4
github.com/ipfs/go-log/v2 v2.1.1
github.com/jackpal/gateway v1.0.5
github.com/jackpal/go-nat-pmp v1.0.2
github.com/jbenet/go-cienv v0.1.0
github.com/jbenet/go-temp-err-catcher v0.1.0
github.com/jbenet/goprocess v0.1.4
github.com/jessevdk/go-flags v1.4.0
github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af
github.com/jpillora/backoff v0.0.0-20180909062703-3050d21c67d7
github.com/jrick/logrotate v1.0.0
github.com/json-iterator/go v1.1.9
github.com/kami-zh/go-capturer v0.0.0-20171211120116-e492ea43421d
github.com/kisielk/errcheck v1.2.0
github.com/kisielk/gotool v1.0.0
github.com/kkdai/bstream v0.0.0-20161212061736-f391b8402d23
github.com/konsorten/go-windows-terminal-sequences v1.0.1
github.com/koron/go-ssdp v0.0.0-20191105050749-2e1c40ed0b5d
github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515
github.com/kr/pretty v0.2.0
github.com/kr/pty v1.1.1
github.com/kr/text v0.1.0
github.com/leodido/go-urn v1.2.0
github.com/libp2p/go-addr-util v0.0.2
github.com/libp2p/go-buffer-pool v0.0.2
github.com/libp2p/go-conn-security-multistream v0.2.0
github.com/libp2p/go-eventbus v0.2.1
github.com/libp2p/go-flow-metrics v0.0.3
github.com/libp2p/go-libp2p v0.11.0
github.com/libp2p/go-libp2p-autonat v0.3.2
github.com/libp2p/go-libp2p-blankhost v0.2.0
github.com/libp2p/go-libp2p-circuit v0.3.1
github.com/libp2p/go-libp2p-connmgr v0.2.4
github.com/libp2p/go-libp2p-core v0.6.1
github.com/libp2p/go-libp2p-crypto v0.1.0
github.com/libp2p/go-libp2p-discovery v0.5.0
github.com/libp2p/go-libp2p-loggables v0.1.0
github.com/libp2p/go-libp2p-mplex v0.2.4
github.com/libp2p/go-libp2p-nat v0.0.6
github.com/libp2p/go-libp2p-netutil v0.1.0
github.com/libp2p/go-libp2p-noise v0.1.1
github.com/libp2p/go-libp2p-peer v0.2.0
github.com/libp2p/go-libp2p-peerstore v0.2.6
github.com/libp2p/go-libp2p-pnet v0.2.0
github.com/libp2p/go-libp2p-pubsub v0.3.5
github.com/libp2p/go-libp2p-secio v0.2.2
github.com/libp2p/go-libp2p-swarm v0.2.8
github.com/libp2p/go-libp2p-testing v0.1.1
github.com/libp2p/go-libp2p-tls v0.1.3
github.com/libp2p/go-libp2p-transport-upgrader v0.3.0
github.com/libp2p/go-libp2p-yamux v0.2.8
github.com/libp2p/go-maddr-filter v0.1.0
github.com/libp2p/go-mplex v0.1.2
github.com/libp2p/go-msgio v0.0.6
github.com/libp2p/go-nat v0.0.5
github.com/libp2p/go-netroute v0.1.3
github.com/libp2p/go-openssl v0.0.7
github.com/libp2p/go-reuseport v0.0.2
github.com/libp2p/go-reuseport-transport v0.0.4
github.com/libp2p/go-sockaddr v0.0.2
github.com/libp2p/go-stream-muxer v0.0.1
github.com/libp2p/go-stream-muxer-multistream v0.3.0
github.com/libp2p/go-tcp-transport v0.2.1
github.com/libp2p/go-ws-transport v0.3.1
github.com/libp2p/go-yamux v1.3.7
github.com/magiconair/properties v1.8.0
github.com/mailru/easyjson v0.0.0-20180823135443-60711f1a8329
github.com/mattn/go-colorable v0.1.2
github.com/mattn/go-isatty v0.0.12
github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b
github.com/miekg/dns v1.1.31
github.com/minio/blake2b-simd v0.0.0-20160723061019-3f5f724cb5b1
github.com/minio/sha256-simd v0.1.1
github.com/mitchellh/go-homedir v1.1.0
github.com/mitchellh/mapstructure v1.1.2
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421
github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742
github.com/mr-tron/base58 v1.2.0
github.com/multiformats/go-base32 v0.0.3
github.com/multiformats/go-base36 v0.1.0
github.com/multiformats/go-multiaddr v0.3.1
github.com/multiformats/go-multiaddr-dns v0.2.0
github.com/multiformats/go-multiaddr-fmt v0.1.0
github.com/multiformats/go-multiaddr-net v0.2.0
github.com/multiformats/go-multibase v0.0.3
github.com/multiformats/go-multihash v0.0.14
github.com/multiformats/go-multistream v0.1.2
github.com/multiformats/go-varint v0.0.6
github.com/nbio/st v0.0.0-20140626010706-e9e8d9816f32
github.com/nxadm/tail v1.4.4
github.com/onsi/ginkgo v1.12.1
github.com/onsi/gomega v1.9.0
github.com/opentracing/opentracing-go v1.2.0
github.com/otiai10/curr v0.0.0-20150429015615-9b4961190c95
github.com/otiai10/mint v1.2.4
github.com/otiai10/primes v0.0.0-20180210170552-f6d2a1ba97c4
github.com/pelletier/go-toml v1.2.0
github.com/pkg/errors v0.9.1
github.com/pmezard/go-difflib v1.0.0
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4
github.com/rogpeppe/fastuuid v1.1.0
github.com/rogpeppe/go-internal v1.3.0
github.com/russross/blackfriday v1.5.2
github.com/sergi/go-diff v1.0.0
github.com/sirupsen/logrus v1.2.0
github.com/smartystreets/assertions v1.0.0
github.com/smartystreets/go-aws-auth v0.0.0-20180515143844-0c1422d1fdb9
github.com/smartystreets/gunit v1.0.0
github.com/smola/gocompat v0.2.0
github.com/spacemonkeygo/openssl v0.0.0-20181017203307-c2dcc5cca94a
github.com/spacemonkeygo/spacelog v0.0.0-20180420211403-2296661a0572
github.com/spaolacci/murmur3 v1.1.0
github.com/spf13/afero v1.1.2
github.com/spf13/cast v1.3.0
github.com/spf13/cobra v0.0.5
github.com/spf13/jwalterweatherman v1.0.0
github.com/spf13/pflag v1.0.3
github.com/spf13/viper v1.3.2
github.com/src-d/envconfig v1.0.0
github.com/stretchr/objx v0.1.1
github.com/stretchr/testify v1.6.1
github.com/syndtr/goleveldb v1.0.0
github.com/tj/assert v0.0.3
github.com/tj/go-buffer v1.1.0
github.com/tj/go-elastic v0.0.0-20171221160941-36157cbbebc2
github.com/tj/go-kinesis v0.0.0-20171128231115-08b17f58cb1b
github.com/tj/go-spin v1.1.0
github.com/ugorji/go v1.1.7
github.com/ugorji/go/codec v1.1.7
github.com/whyrusleeping/go-keyspace v0.0.0-20160322163242-5b898ac5add1
github.com/whyrusleeping/go-logging v0.0.1
github.com/whyrusleeping/mafmt v1.2.8
github.com/whyrusleeping/mdns v0.0.0-20190826153040-b9b60ed33aa9
github.com/whyrusleeping/multiaddr-filter v0.0.0-20160516205228-e903e4adabd7
github.com/whyrusleeping/timecache v0.0.0-20160911033111-cfcb2f1abfee
github.com/x-cray/logrus-prefixed-formatter v0.5.2
github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77
github.com/yuin/goldmark v1.2.1
go.opencensus.io v0.22.4
go.uber.org/atomic v1.6.0
go.uber.org/goleak v1.0.0
go.uber.org/multierr v1.5.0
go.uber.org/tools v0.0.0-20190618225709-2cfd321de3ee
go.uber.org/zap v1.15.0
golang.org/x/crypto v0.0.0-20200820211705-5c72a883971a
golang.org/x/exp v0.0.0-20190121172915-509febef88a4
golang.org/x/lint v0.0.0-20190930215403-16217165b5de
golang.org/x/mod v0.3.0
golang.org/x/net v0.0.0-20200822124328-c89045814202
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be
golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208
golang.org/x/sys v0.0.0-20200828081204-131dc92a58d5
golang.org/x/text v0.3.2
golang.org/x/tools v0.0.0-20200925191224-5d1fdd8fa346
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1
google.golang.org/appengine v1.4.0
google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013
google.golang.org/grpc v1.27.0
google.golang.org/protobuf v1.25.0
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15
gopkg.in/eapache/go-resiliency.v1 v1.2.0
gopkg.in/errgo.v2 v2.1.0
gopkg.in/fsnotify.v1 v1.4.7
gopkg.in/h2non/gentleman-retry.v2 v2.0.1
gopkg.in/h2non/gentleman.v2 v2.0.4
gopkg.in/src-d/go-cli.v0 v0.0.0-20181105080154-d492247bbc0d
gopkg.in/src-d/go-log.v1 v1.0.1
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7
gopkg.in/yaml.v2 v2.2.8
gopkg.in/yaml.v3 v3.0.0-20200605160147-a5ece683394c
honnef.co/go/tools v0.0.1-2019.2.3
aschmahmann commented 4 years ago

@Strernd as a sanity check could you make sure this is actually a libp2p issue instead of a Docker config one?

For example, instead of doing a host.Connect() on your manually discovered peer's address just do a direct TCP connect to it such as:

    conn, err := net.Dial("tcp", "172.31.37.27:5500")
    if err != nil {
        panic(err)
    }
    fmt.Println(conn.RemoteAddr())
    return

If you're concerned about the libp2p server side you could replace that with a go TCP listener as well, such as:

 l, err := net.Listen("tcp", 5500)
        if err != nil {
                fmt.Println(err)
                return
        }
        defer l.Close()

        c, err := l.Accept()
        if err != nil {
                fmt.Println(err)
                return
        }
Strernd commented 4 years ago

Will do it tomorrow!

Strernd commented 4 years ago

I finally came around to test the code, it took me a while. I also tested if the ports are open the following way: On one instance nc -l <internal_ip> 5500 and on the other instance nc -l <other_inst_ip> 5500 which was successful in creating a connection, while other random ports weren't. The docker-compose files each have a entry - "5500:5500" under ports.

I tried to connect with net.Dial which fails with dial tcp <other_peer_interal_ip>:5500: connect: connection refused

I also tried to change to addressFactory such that it just returns the fixed address and port.

log.WithFields(log.Fields{"addresses": host.Addrs(), "id": host.ID()}).Info("Listening") logs addresses=[/ip4/<nodes_internal_ip>/tcp/5500] id=Qm... with the said change to the addressFactory.

Edit: I also implemented net as server now. Connections can be established. So either I'm using libp2p wrong, or something doesn't work as it's intended to.

Strernd commented 4 years ago

For future reference, here is what I had to change to fix this:

A bit counter-intuitive for me.