search

hyperledger / cello

Operating System for Enterprise Blockchain
https://wiki.hyperledger.org/display/cello
Apache License 2.0
897 stars 445 forks source link

cello v0.9.0-h3c, error log: TLS handshake failed with error EOF server=Orderer remoteaddress=172.24.0.1:59598 #587

Open BlackFlame33 opened 11 months ago

BlackFlame33 commented 11 months ago

Sorry, my English is very bad so I translate my word to English using DeepL and put Chinese word below.

Hyperledger Fabric v1.4, cello v0.9.0-h3c, AlmaLinux 8.7

Hello, When I follow this tutorial and cello's docs and get to the step of creating the network, the node starts successfully, but the logs always show a TLS handshake failure. This is the error log for the orderer node:

大家好，当我按照这份教程和cello官方的文档，到创建网络这一步节点启动成功了，但是日志里总是会显示TLS握手失败。这是orderer节点的报错日志：

image.png

This is the error log for the peer node:

这是peer节点的报错日志：

image.png

Where 172.24.0.1 is the gateway address in the celloNet network created by docker(or cello perhaps).

其中，172.24.0.1是docker创建的celloNet网络中网关地址。

image.png

My question is, why is it that if I follow the tutorial step by step, it still reports a TLS handshake failure? Is this "TLS Handshake Failure" a failure to connect between nodes or a failure to connect to cello? What should I do about it? Does this affect the secure communication of the blockchain network?

我的疑问是，为什么完全按照教程一步步走还是会报TLS握手失败？这个“TLS握手失败”是节点之间连接失败还是与cello连接失败？我应该如何处理？这会影响到区块链网络的安全通信吗？

My guess is that if it's a connection failure between nodes, the logs of each node should show a peer's ip, but all the logs of all the nodes show 172.24.0.1, which is the gateway address of celloNet network, so what does it mean?

我的猜想是，如果是节点之间连接失败，应该各节点的日志报错会显示节点的ip，但所有节点的日志报错都显示的是172.24.0.1，这是celloNet网络的网关地址，它意味着什么？

BlackFlame33 commented 11 months ago

My cello configuration is like this: image.png

BlackFlame33 commented 11 months ago

Hello! After yesterday's exploration, I also did the following steps:

  1. I noticed that README-H3C.md mentions the need to synchronize the time between the host and the worker, so I adjusted the time zones of all the containers to be all the same (to avoid an 8 hour difference)
  2. README-H3C.md also mentions that if you are using docker, you can't choose etcdraft as the consensus algorithm, so I created a new network to use solo consensus.
  3. If you choose couchDB for the database, you will get an error, and it will tell you that it can't find "hyperledger/fabric-couchdb:2.1.1", but I can see that the latest version in the official docker repository is 0.4. So I'm not quite sure if fabric-couchdb:2.1.1 requires a local build or is no longer available, I'm currently using leveldb.

大家好，经过昨天的探索，我还做了以下步骤：

  1. 我注意到README-H3C.md提到需要主机与worker之间的时间要同步，所以我将所有容器的时区全部调整成一致的（避免相差8小时）
  2. README-H3C.md还提到如果是使用docker的话，共识算法不能选择etcdraft，所以我新建了一个网络使用solo共识
  3. 如果数据库选择couchDB会报错，它会提示说找不到"hyperledger/fabric-couchdb:2.1.1"，但我在官方docker仓库里看到最新版本是到0.4的。所以我不太清楚fabric-couchdb:2.1.1是需要本地构建还是已经无法获取，目前使用的时leveldb。 
    
[2023-11-29 09:50:10,060] INFO [modules.blockchain_network] [blockchain_network.py:496               create()] -  before function file_define.commad_create_path,and path is
[2023-11-29 09:50:10,061] INFO [common.fabric_network_define] [fabric_network_define.py:57   commad_create_path()] - before commad_create_path: /opt/fabric/8fedb0231daf4523a97c65404b6b7bfa
[2023-11-29 09:50:10,094] INFO [common.fabric_network_define] [fabric_network_define.py:65   commad_create_path()] -  is = True
[2023-11-29 09:50:10,095] INFO [modules.blockchain_network] [blockchain_network.py:504               create()] -  after function file_define.commad_create_path,and path is /opt/fabric/8fedb0231daf4523a97c65404b6b7bfa
org3.example.com
org4.example.com
2023-11-29 09:50:10.219 CST [common.tools.configtxgen] main -> WARN 001 Omitting the channel ID for configtxgen for output operations is deprecated.  Explicitly passing the channel ID will be required in the future, defaulting to 'testchainid'.
2023-11-29 09:50:10.219 CST [common.tools.configtxgen] main -> INFO 002 Loading configuration
2023-11-29 09:50:10.221 CST [common.tools.configtxgen.localconfig] completeInitialization -> INFO 003 orderer type: solo
2023-11-29 09:50:10.221 CST [common.tools.configtxgen.localconfig] Load -> INFO 004 Loaded configuration: /opt/fabric/8fedb0231daf4523a97c65404b6b7bfa/configtx.yaml
2023-11-29 09:50:10.221 CST [common.tools.configtxgen.localconfig] completeInitialization -> INFO 005 orderer type: solo
2023-11-29 09:50:10.221 CST [common.tools.configtxgen.localconfig] LoadTopLevel -> INFO 006 Loaded configuration: /opt/fabric/8fedb0231daf4523a97c65404b6b7bfa/configtx.yaml
2023-11-29 09:50:10.221 CST [common.tools.configtxgen.encoder] NewChannelGroup -> WARN 007 Default policy emission is deprecated, please include policy specifications for the channel group in configtx.yaml
2023-11-29 09:50:10.221 CST [common.tools.configtxgen.encoder] NewOrdererGroup -> WARN 008 Default policy emission is deprecated, please include policy specifications for the orderer group in configtx.yaml
2023-11-29 09:50:10.222 CST [common.tools.configtxgen.encoder] NewOrdererOrgGroup -> WARN 009 Default policy emission is deprecated, please include policy specifications for the orderer org group Orderer2Org in configtx.yaml
2023-11-29 09:50:10.222 CST [common.tools.configtxgen.encoder] NewConsortiumOrgGroup -> WARN 00a Default policy emission is deprecated, please include policy specifications for the orderer org group Org3MSP in configtx.yaml
2023-11-29 09:50:10.222 CST [common.tools.configtxgen.encoder] NewConsortiumOrgGroup -> WARN 00b Default policy emission is deprecated, please include policy specifications for the orderer org group Org4MSP in configtx.yaml
2023-11-29 09:50:10.222 CST [common.tools.configtxgen] doOutputBlock -> INFO 00c Generating genesis block
2023-11-29 09:50:10.222 CST [common.tools.configtxgen] doOutputBlock -> INFO 00d Writing genesis block
[2023-11-29 09:50:10,252] WARNING [agent.docker.docker_swarm] [docker_swarm.py:161         check_daemon()] - invalid workder_api={}
[2023-11-29 09:50:10,253] WARNING [modules.host] [host.py:405       refresh_status()] - Host 515e20888ffc4e389786d5e2c5744892 is inactive
[2023-11-29 09:50:10,270] INFO [resources.blockchain_network_api] [blockchain_network_api.py:223 blockchain_network_list()] - /blockchain_network method=GET
[2023-11-29 09:50:10,271] INFO [modules.blockchain_network] [blockchain_network.py:815                 list()] - filter data {}
Creating 8fedb0231daf_orderer2-orderer2 ... 
Creating 8fedb0231daf_orderer1-orderer2 ... 
Creating 8fedb0231daf_orderer3-orderer2 ...

Creating 8fedb0231daf_orderer2-orderer2 ... done [2023-11-29 09:50:13,145] INFO [modules.blockchain_network] [blockchain_network.py:230 get_endpoints_list()] - filter data 8fedb0231daf4523a97c65404b6b7bfa

Creating 8fedb0231daf_orderer1-orderer2 ... done

Creating 8fedb0231daf_orderer3-orderer2 ... done [2023-11-29 09:50:13,272] INFO [resources.organization_api] [organization_api.py:279 organization_list()] - /organization_list method=GET [2023-11-29 09:50:13,272] INFO [modules.organization] [organization.py:59 list()] - filter data {} [2023-11-29 09:50:18,267] WARNING [agent.docker.docker_swarm] [docker_swarm.py:161 check_daemon()] - invalid workder_api={} [2023-11-29 09:50:18,267] WARNING [modules.host] [host.py:405 refresh_status()] - Host 515e20888ffc4e389786d5e2c5744892 is inactive Found orphan containers (8fedb0231daf_orderer2-orderer2, 8fedb0231daf_orderer1-orderer2, 8fedb0231daf_orderer3-orderer2) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up. [2023-11-29 09:50:22,747] ERROR [modules.blockchain_network] [blockchain_network.py:328 _create_network()] - network 8fedb0231daf4523a97c65404b6b7bfa create failed for 404 Client Error: Not Found ("manifest for hyperledger/fabric-couchdb:2.1.1 not found: manifest unknown: manifest unknown") [2023-11-29 09:50:22,748] INFO [modules.blockchain_network] [blockchain_network.py:170 delete()] - remove network from host, network:8fedb0231daf4523a97c65404b6b7bfa [2023-11-29 09:50:22,751] WARNING [agent.docker.docker_swarm] [docker_swarm.py:161 check_daemon()] - invalid workder_api={} [2023-11-29 09:50:22,751] WARNING [modules.host] [host.py:405 refresh_status()] - Host 515e20888ffc4e389786d5e2c5744892 is inactive Found orphan containers (8fedb0231daf_orderer2-orderer2, 8fedb0231daf_orderer1-orderer2, 8fedb0231daf_orderer3-orderer2) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up. [2023-11-29 09:50:22,828] INFO [modules.blockchain_network] [blockchain_network.py:204 delete()] - remove network 8fedb0231daf4523a97c65404b6b7bfa fail from host Exception in thread Thread-51: Traceback (most recent call last): File "/usr/local/lib/python3.5/dist-packages/docker/api/client.py", line 222, in _raise_for_status response.raise_for_status() File "/usr/local/lib/python3.5/dist-packages/requests/models.py", line 909, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http://192.168.206.128:2375/v1.25/images/create?fromImage=hyperledger%2Ffabric-couchdb&tag=2.1.1 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/app/modules/blockchain_network.py", line 309, in _create_network fabric_version, request_host_ports, portid, peer_num) File "/app/agent/docker/blockchain_network.py", line 376, in create_peer_org containers = project.up(detached=True, timeout=5) File "/usr/local/lib/python3.5/dist-packages/compose/project.py", line 452, in up svc.ensure_image_exists(do_build=do_build) File "/usr/local/lib/python3.5/dist-packages/compose/service.py", line 318, in ensure_image_exists self.pull() File "/usr/local/lib/python3.5/dist-packages/compose/service.py", line 1074, in pull output = self.client.pull(repo, tag=tag, stream=True) File "/usr/local/lib/python3.5/dist-packages/docker/api/image.py", line 393, in pull self._raise_for_status(response) File "/usr/local/lib/python3.5/dist-packages/docker/api/client.py", line 224, in _raise_for_status raise create_api_error_from_http_exception(e) File "/usr/local/lib/python3.5/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception raise cls(e, response=response, explanation=explanation) docker.errors.NotFound: 404 Client Error: Not Found ("manifest for hyperledger/fabric-couchdb:2.1.1 not found: manifest unknown: manifest unknown") During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.5/dist-packages/docker/api/client.py", line 222, in _raise_for_status response.raise_for_status() File "/usr/local/lib/python3.5/dist-packages/requests/models.py", line 909, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: http://192.168.206.128:2375/v1.25/networks/8fedb0231daf_celloNet During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner self.run() File "/usr/lib/python3.5/threading.py", line 862, in run self._target(*self._args, self._kwargs) File "/app/modules/blockchain_network.py", line 330, in _create_network self.delete(network) File "/app/modules/blockchain_network.py", line 206, in delete raise e File "/app/modules/blockchain_network.py", line 179, in delete self.host_agents[host.type].delete_peer_org(peer_org, host, net_id) File "/app/agent/docker/blockchain_network.py", line 460, in delete_peer_org project.down(ImageType_none, True) File "/usr/local/lib/python3.5/dist-packages/compose/project.py", line 338, in down self.networks.remove() File "/usr/local/lib/python3.5/dist-packages/compose/network.py", line 252, in remove network.remove() File "/usr/local/lib/python3.5/dist-packages/compose/network.py", line 92, in remove self.client.remove_network(self.full_name) File "/usr/local/lib/python3.5/dist-packages/docker/utils/decorators.py", line 34, in wrapper return f(self, *args, *kwargs) File "/usr/local/lib/python3.5/dist-packages/docker/utils/decorators.py", line 19, in wrapped return f(self, resource_id, args, kwargs) File "/usr/local/lib/python3.5/dist-packages/docker/api/network.py", line 189, in remove_network self._raise_for_status(res) File "/usr/local/lib/python3.5/dist-packages/docker/api/client.py", line 224, in _raise_for_status raise create_api_error_from_http_exception(e) File "/usr/local/lib/python3.5/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception raise cls(e, response=response, explanation=explanation) docker.errors.APIError: 403 Client Error: Forbidden ("error while removing network: network 8fedb0231daf_celloNet id 1156402bec6790322f21442faba9dadd739c7b3df553cab58f0c6b734b20636f has active endpoints") [2023-11-29 09:50:23,547] INFO [modules.blockchain_network] [blockchain_network.py:230 get_endpoints_list()] - filter data 8fedb0231daf4523a97c65404b6b7bfa


After completing the above steps, I created the channel, installed the chaincode, instantiated the chaincode, and on the last attempt at cello, the button would keep spinning around after clicking on the instantiate chaincode. This time, instantiating the chaincode worked, and the invoke and query operations can be performed, but the node logs will always show a TLS handshake failure:

完成以上步骤后，我创建通道，安装链码，实例化链码，在上次尝试cello时，点击实例化链码后按钮会一直转圈。这次实例化链码成功了，可以进行invoke和query操作，但节点日志中始终会显示TLS握手失败：
![image.png](https://s2.loli.net/2023/11/29/XdE9tov3a5YGQcr.png)

This error has always bothered me, and I can't tell if this represents a failed TLS handshake between nodes, or a failed TLS handshake between cello and the blockchain network.
这个报错始终困扰着我，我无法判断这到底代表节点之间的TLS握手失败，还是cello与区块链网络之间的TLS握手失败。
fengyangsy commented 11 months ago

可以确定这个是节点之间的握手失败，可能是gossip的握手，曾经优化过这个问题，具体记不清了。应该不会影响正常交易。

BlackFlame33 commented 11 months ago

可以确定这个是节点之间的握手失败，可能是gossip的握手，曾经优化过这个问题，具体记不清了。应该不会影响正常交易。

你好！感谢回复。请问目前有解决方案吗？我看了Fabric官方有关gossip的文档。gossip握手失败的话难道不会导致区块链网络不安全不可信吗？已知peer，orderer，ca节点日志全部都会报这个TLS握手失败。网络部署的docker-compose文件的环境变量是这样的：

networks: {celloNet: null}
services:
  ca.org1.h3c.com:
    command: sh -c 'fabric-ca-server start -b admin:adminpw -d                --config
      /etc/hyperledger/fabric-ca-server-config/fabric-ca-server-config.yaml'
    container_name: 41618d07d193_ca.org1.h3c.com
    environment: [FABRIC_CA_HOME=/etc/hyperledger/fabric-ca-server, FABRIC_CA_SERVER_CA_NAME=ca-org1,
      FABRIC_CA_SERVER_CA_CERTFILE=/etc/hyperledger/fabric-ca-server-config/ca.org1.h3c.com-cert.pem,
      FABRIC_CA_SERVER_CA_KEYFILE=/etc/hyperledger/fabric-ca-server-config/d46f1dc8df195f0fd7c683e8bb38acd1f18e987e52c7f700482d6c215da572dc_sk,
      FABRIC_CA_SERVER_TLS_ENABLED=true, FABRIC_CA_SERVER_TLS_CERTFILE=/etc/hyperledger/fabric-ca-server-config/ca.org1.h3c.com-cert.pem,
      FABRIC_CA_SERVER_TLS_KEYFILE=/etc/hyperledger/fabric-ca-server-config/d46f1dc8df195f0fd7c683e8bb38acd1f18e987e52c7f700482d6c215da572dc_sk]
    image: hyperledger/fabric-ca:1.4.2
    networks: [celloNet]
    ports: ['30007:7054']
    volumes: ['/opt/cello/41618d07d1934f25a38cd579f75c67cd/crypto-config/peerOrganizations/org1.h3c.com/ca/:/etc/hyperledger/fabric-ca-server-config']
  ca.org2.h3c.com:
    command: sh -c 'fabric-ca-server start -b admin:adminpw -d                --config
      /etc/hyperledger/fabric-ca-server-config/fabric-ca-server-config.yaml'
    container_name: 41618d07d193_ca.org2.h3c.com
    environment: [FABRIC_CA_HOME=/etc/hyperledger/fabric-ca-server, FABRIC_CA_SERVER_CA_NAME=ca-org2,
      FABRIC_CA_SERVER_CA_CERTFILE=/etc/hyperledger/fabric-ca-server-config/ca.org2.h3c.com-cert.pem,
      FABRIC_CA_SERVER_CA_KEYFILE=/etc/hyperledger/fabric-ca-server-config/36ca3558b302d12ab38b74b5c3487bad3e12d9bcf19d1bd91e6183ffee5990ed_sk,
      FABRIC_CA_SERVER_TLS_ENABLED=true, FABRIC_CA_SERVER_TLS_CERTFILE=/etc/hyperledger/fabric-ca-server-config/ca.org2.h3c.com-cert.pem,
      FABRIC_CA_SERVER_TLS_KEYFILE=/etc/hyperledger/fabric-ca-server-config/36ca3558b302d12ab38b74b5c3487bad3e12d9bcf19d1bd91e6183ffee5990ed_sk]
    image: hyperledger/fabric-ca:1.4.2
    networks: [celloNet]
    ports: ['30012:7054']
    volumes: ['/opt/cello/41618d07d1934f25a38cd579f75c67cd/crypto-config/peerOrganizations/org2.h3c.com/ca/:/etc/hyperledger/fabric-ca-server-config']
  orderer1-orderer:
    command: orderer
    container_name: 41618d07d193_orderer1-orderer
    environment: [ORDERER_GENERAL_LOGLEVEL=DEBUG, ORDERER_GENERAL_LISTENADDRESS=0.0.0.0,
      ORDERER_GENERAL_LISTENPORT=30000, ORDERER_GENERAL_GENESISMETHOD=file, ORDERER_GENERAL_GENESISFILE=/var/hyperledger/orderer/orderer.genesis.block,
      ORDERER_GENERAL_LOCALMSPID=OrdererMSP, ORDERER_GENERAL_LOCALMSPDIR=/var/hyperledger/orderer/msp,
      ORDERER_GENERAL_TLS_ENABLED=true, ORDERER_GENERAL_TLS_PRIVATEKEY=/var/hyperledger/orderer/tls/server.key,
      ORDERER_GENERAL_TLS_CERTIFICATE=/var/hyperledger/orderer/tls/server.crt, 'ORDERER_GENERAL_TLS_ROOTCAS=[/var/hyperledger/orderer/tls/ca.crt]',
      ORDERER_GENERAL_CLUSTER_CLIENTCERTIFICATE=/var/hyperledger/orderer/tls/server.crt,
      ORDERER_GENERAL_CLUSTER_CLIENTPRIVATEKEY=/var/hyperledger/orderer/tls/server.key,
      'ORDERER_GENERAL_CLUSTER_ROOTCAS=[/var/hyperledger/orderer/tls/ca.crt]']
    image: hyperledger/fabric-orderer:1.4.2
    networks: [celloNet]
    ports: ['30000:30000']
    volumes: ['/opt/cello/41618d07d1934f25a38cd579f75c67cd/crypto-config/ordererOrganizations/h3c.com/orderers/orderer1.h3c.com/msp:/var/hyperledger/orderer/msp',
      '/opt/cello/41618d07d1934f25a38cd579f75c67cd/crypto-config/ordererOrganizations/h3c.com/orderers/orderer1.h3c.com/tls:/var/hyperledger/orderer/tls',
      '/opt/cello/41618d07d1934f25a38cd579f75c67cd/channel-artifacts/genesis.block:/var/hyperledger/orderer/orderer.genesis.block']
  orderer2-orderer:
    command: orderer
    container_name: 41618d07d193_orderer2-orderer
    environment: [ORDERER_GENERAL_LOGLEVEL=DEBUG, ORDERER_GENERAL_LISTENADDRESS=0.0.0.0,
      ORDERER_GENERAL_LISTENPORT=30001, ORDERER_GENERAL_GENESISMETHOD=file, ORDERER_GENERAL_GENESISFILE=/var/hyperledger/orderer/orderer.genesis.block,
      ORDERER_GENERAL_LOCALMSPID=OrdererMSP, ORDERER_GENERAL_LOCALMSPDIR=/var/hyperledger/orderer/msp,
      ORDERER_GENERAL_TLS_ENABLED=true, ORDERER_GENERAL_TLS_PRIVATEKEY=/var/hyperledger/orderer/tls/server.key,
      ORDERER_GENERAL_TLS_CERTIFICATE=/var/hyperledger/orderer/tls/server.crt, 'ORDERER_GENERAL_TLS_ROOTCAS=[/var/hyperledger/orderer/tls/ca.crt]',
      ORDERER_GENERAL_CLUSTER_CLIENTCERTIFICATE=/var/hyperledger/orderer/tls/server.crt,
      ORDERER_GENERAL_CLUSTER_CLIENTPRIVATEKEY=/var/hyperledger/orderer/tls/server.key,
      'ORDERER_GENERAL_CLUSTER_ROOTCAS=[/var/hyperledger/orderer/tls/ca.crt]']
    image: hyperledger/fabric-orderer:1.4.2
    networks: [celloNet]
    ports: ['30001:30001']
    volumes: ['/opt/cello/41618d07d1934f25a38cd579f75c67cd/crypto-config/ordererOrganizations/h3c.com/orderers/orderer2.h3c.com/msp:/var/hyperledger/orderer/msp',
      '/opt/cello/41618d07d1934f25a38cd579f75c67cd/crypto-config/ordererOrganizations/h3c.com/orderers/orderer2.h3c.com/tls:/var/hyperledger/orderer/tls',
      '/opt/cello/41618d07d1934f25a38cd579f75c67cd/channel-artifacts/genesis.block:/var/hyperledger/orderer/orderer.genesis.block']
  orderer3-orderer:
    command: orderer
    container_name: 41618d07d193_orderer3-orderer
    environment: [ORDERER_GENERAL_LOGLEVEL=DEBUG, ORDERER_GENERAL_LISTENADDRESS=0.0.0.0,
      ORDERER_GENERAL_LISTENPORT=30002, ORDERER_GENERAL_GENESISMETHOD=file, ORDERER_GENERAL_GENESISFILE=/var/hyperledger/orderer/orderer.genesis.block,
      ORDERER_GENERAL_LOCALMSPID=OrdererMSP, ORDERER_GENERAL_LOCALMSPDIR=/var/hyperledger/orderer/msp,
      ORDERER_GENERAL_TLS_ENABLED=true, ORDERER_GENERAL_TLS_PRIVATEKEY=/var/hyperledger/orderer/tls/server.key,
      ORDERER_GENERAL_TLS_CERTIFICATE=/var/hyperledger/orderer/tls/server.crt, 'ORDERER_GENERAL_TLS_ROOTCAS=[/var/hyperledger/orderer/tls/ca.crt]',
      ORDERER_GENERAL_CLUSTER_CLIENTCERTIFICATE=/var/hyperledger/orderer/tls/server.crt,
      ORDERER_GENERAL_CLUSTER_CLIENTPRIVATEKEY=/var/hyperledger/orderer/tls/server.key,
      'ORDERER_GENERAL_CLUSTER_ROOTCAS=[/var/hyperledger/orderer/tls/ca.crt]']
    image: hyperledger/fabric-orderer:1.4.2
    networks: [celloNet]
    ports: ['30002:30002']
    volumes: ['/opt/cello/41618d07d1934f25a38cd579f75c67cd/crypto-config/ordererOrganizations/h3c.com/orderers/orderer3.h3c.com/msp:/var/hyperledger/orderer/msp',
      '/opt/cello/41618d07d1934f25a38cd579f75c67cd/crypto-config/ordererOrganizations/h3c.com/orderers/orderer3.h3c.com/tls:/var/hyperledger/orderer/tls',
      '/opt/cello/41618d07d1934f25a38cd579f75c67cd/channel-artifacts/genesis.block:/var/hyperledger/orderer/orderer.genesis.block']
  peer0.org1.h3c.com:
    command: peer node start
    container_name: 41618d07d193_peer0.org1.h3c.com
    environment: [CORE_PEER_ID=peer0.org1.h3c.com, CORE_PEER_LOCALMSPID=Org1MSP, 'CORE_PEER_ADDRESS=peer0.org1.h3c.com:7051',
      CORE_VM_DOCKER_HOSTCONFIG_NETWORKMODE=41618d07d193_celloNet, CORE_LOGGING_LEVEL=DEBUG,
      CORE_PEER_GOSSIP_USELEADERELECTION=true, CORE_PEER_GOSSIP_ORGLEADER=false, CORE_PEER_GOSSIP_SKIPHANDSHAKE=true,
      CORE_PEER_TLS_ENABLED=true, CORE_PEER_TLS_CERT_FILE=/etc/hyperledger/fabric/tls/server.crt,
      CORE_PEER_TLS_KEY_FILE=/etc/hyperledger/fabric/tls/server.key, CORE_PEER_TLS_ROOTCERT_FILE=/etc/hyperledger/fabric/tls/ca.crt,
      GODEBUG=netdns=go]
    image: hyperledger/fabric-peer:1.4.2
    networks: [celloNet]
    ports: ['30003:7051', '30004:7052']
    volumes: ['/var/run/:/var/run/', '/opt/cello/41618d07d1934f25a38cd579f75c67cd/crypto-config/peerOrganizations/org1.h3c.com/peers/peer0.org1.h3c.com/msp:/etc/hyperledger/fabric/msp',
      '/opt/cello/41618d07d1934f25a38cd579f75c67cd/crypto-config/peerOrganizations/org1.h3c.com/peers/peer0.org1.h3c.com/tls:/etc/hyperledger/fabric/tls']
  peer0.org2.h3c.com:
    command: peer node start
    container_name: 41618d07d193_peer0.org2.h3c.com
    environment: [CORE_PEER_ID=peer0.org2.h3c.com, CORE_PEER_LOCALMSPID=Org2MSP, 'CORE_PEER_ADDRESS=peer0.org2.h3c.com:7051',
      CORE_VM_DOCKER_HOSTCONFIG_NETWORKMODE=41618d07d193_celloNet, CORE_LOGGING_LEVEL=DEBUG,
      CORE_PEER_GOSSIP_USELEADERELECTION=true, CORE_PEER_GOSSIP_ORGLEADER=false, CORE_PEER_GOSSIP_SKIPHANDSHAKE=true,
      CORE_PEER_TLS_ENABLED=true, CORE_PEER_TLS_CERT_FILE=/etc/hyperledger/fabric/tls/server.crt,
      CORE_PEER_TLS_KEY_FILE=/etc/hyperledger/fabric/tls/server.key, CORE_PEER_TLS_ROOTCERT_FILE=/etc/hyperledger/fabric/tls/ca.crt,
      GODEBUG=netdns=go]
    image: hyperledger/fabric-peer:1.4.2
    networks: [celloNet]
    ports: ['30008:7051', '30009:7052']
    volumes: ['/var/run/:/var/run/', '/opt/cello/41618d07d1934f25a38cd579f75c67cd/crypto-config/peerOrganizations/org2.h3c.com/peers/peer0.org2.h3c.com/msp:/etc/hyperledger/fabric/msp',
      '/opt/cello/41618d07d1934f25a38cd579f75c67cd/crypto-config/peerOrganizations/org2.h3c.com/peers/peer0.org2.h3c.com/tls:/etc/hyperledger/fabric/tls']
  peer1.org1.h3c.com:
    command: peer node start
    container_name: 41618d07d193_peer1.org1.h3c.com
    environment: [CORE_PEER_ID=peer1.org1.h3c.com, CORE_PEER_LOCALMSPID=Org1MSP, 'CORE_PEER_ADDRESS=peer1.org1.h3c.com:7051',
      CORE_VM_DOCKER_HOSTCONFIG_NETWORKMODE=41618d07d193_celloNet, CORE_LOGGING_LEVEL=DEBUG,
      CORE_PEER_GOSSIP_USELEADERELECTION=true, CORE_PEER_GOSSIP_ORGLEADER=false, CORE_PEER_GOSSIP_SKIPHANDSHAKE=true,
      CORE_PEER_TLS_ENABLED=true, CORE_PEER_TLS_CERT_FILE=/etc/hyperledger/fabric/tls/server.crt,
      CORE_PEER_TLS_KEY_FILE=/etc/hyperledger/fabric/tls/server.key, CORE_PEER_TLS_ROOTCERT_FILE=/etc/hyperledger/fabric/tls/ca.crt,
      GODEBUG=netdns=go]
    image: hyperledger/fabric-peer:1.4.2
    networks: [celloNet]
    ports: ['30005:7051', '30006:7052']
    volumes: ['/var/run/:/var/run/', '/opt/cello/41618d07d1934f25a38cd579f75c67cd/crypto-config/peerOrganizations/org1.h3c.com/peers/peer1.org1.h3c.com/msp:/etc/hyperledger/fabric/msp',
      '/opt/cello/41618d07d1934f25a38cd579f75c67cd/crypto-config/peerOrganizations/org1.h3c.com/peers/peer1.org1.h3c.com/tls:/etc/hyperledger/fabric/tls']
  peer1.org2.h3c.com:
    command: peer node start
    container_name: 41618d07d193_peer1.org2.h3c.com
    environment: [CORE_PEER_ID=peer1.org2.h3c.com, CORE_PEER_LOCALMSPID=Org2MSP, 'CORE_PEER_ADDRESS=peer1.org2.h3c.com:7051',
      CORE_VM_DOCKER_HOSTCONFIG_NETWORKMODE=41618d07d193_celloNet, CORE_LOGGING_LEVEL=DEBUG,
      CORE_PEER_GOSSIP_USELEADERELECTION=true, CORE_PEER_GOSSIP_ORGLEADER=false, CORE_PEER_GOSSIP_SKIPHANDSHAKE=true,
      CORE_PEER_TLS_ENABLED=true, CORE_PEER_TLS_CERT_FILE=/etc/hyperledger/fabric/tls/server.crt,
      CORE_PEER_TLS_KEY_FILE=/etc/hyperledger/fabric/tls/server.key, CORE_PEER_TLS_ROOTCERT_FILE=/etc/hyperledger/fabric/tls/ca.crt,
      GODEBUG=netdns=go]
    image: hyperledger/fabric-peer:1.4.2
    networks: [celloNet]
    ports: ['30010:7051', '30011:7052']
    volumes: ['/var/run/:/var/run/', '/opt/cello/41618d07d1934f25a38cd579f75c67cd/crypto-config/peerOrganizations/org2.h3c.com/peers/peer1.org2.h3c.com/msp:/etc/hyperledger/fabric/msp',
      '/opt/cello/41618d07d1934f25a38cd579f75c67cd/crypto-config/peerOrganizations/org2.h3c.com/peers/peer1.org2.h3c.com/tls:/etc/hyperledger/fabric/tls']
version: '3.2'

这里面跟gossip相关的环境变量的配置：CORE_PEER_GOSSIP_USELEADERELECTION, CORE_PEER_GOSSIP_ORGLEADER, CORE_PEER_GOSSIP_SKIPHANDSHAKE是否会是导致问题的主要原因呢？

xudley commented 3 months ago

I encountered a similar issue with TLS handshake failures when deploying a Hyperledger Fabric network using Cello. The failed requests were originating from the Docker gateway. After some attempts, I found that the issue seemed resolved when I manually deployed the network using the crypto-config.yaml, configtx.yaml, and docker-compose.yaml files generated by Cello. Here’s what I did:

Updated the container_name in docker-compose.yaml to match the SANS names in crypto-config.yaml. Upgraded the Orderer and Peer images to 1.4.12, and the CA image to 1.4.9. Regenerated crypto-config and genesis.block using Fabric tools (v1.4.12). This led me to suspect that the issue might be related to DNS resolution problems within the Docker network. It appears that nodes were not communicating directly but were trying to connect through the Docker gateway, which could explain the TLS handshake failures.

Is this a viable solution? What potential issues could this cause? crypto-config.yaml image configtx.yaml image

networks: { celloNet: null }
services:
  ca.org2.example.com:
    command:
      sh -c 'fabric-ca-server start -b admin:adminpw -d                --config
      /etc/hyperledger/fabric-ca-server-config/fabric-ca-server-config.yaml'
    container_name: ca.org2.example.com
    environment:
      [
        FABRIC_CA_HOME=/etc/hyperledger/fabric-ca-server,
        FABRIC_CA_SERVER_CA_NAME=ca-org2,
        FABRIC_CA_SERVER_CA_CERTFILE=/etc/hyperledger/fabric-ca-server-config/ca.org2.example.com-cert.pem,
        FABRIC_CA_SERVER_CA_KEYFILE=/etc/hyperledger/fabric-ca-server-config/a6c2302214d375db89d9a66d6cfb888a6b351692d4f14a927ea1c2e155b08c9b_sk,
        FABRIC_CA_SERVER_TLS_ENABLED=true,
        FABRIC_CA_SERVER_TLS_CERTFILE=/etc/hyperledger/fabric-ca-server-config/ca.org2.example.com-cert.pem,
        FABRIC_CA_SERVER_TLS_KEYFILE=/etc/hyperledger/fabric-ca-server-config/a6c2302214d375db89d9a66d6cfb888a6b351692d4f14a927ea1c2e155b08c9b_sk,
      ]
    image: hyperledger/fabric-ca:1.4.9
    networks: [celloNet]
    ports: ["30003:7054"]
    volumes:
      [
        "../crypto-config/peerOrganizations/org2.example.com/ca/:/etc/hyperledger/fabric-ca-server-config",
      ]
  ca.org3.example.com:
    command:
      sh -c 'fabric-ca-server start -b admin:adminpw -d                --config
      /etc/hyperledger/fabric-ca-server-config/fabric-ca-server-config.yaml'
    container_name: ca.org3.example.com
    environment:
      [
        FABRIC_CA_HOME=/etc/hyperledger/fabric-ca-server,
        FABRIC_CA_SERVER_CA_NAME=ca-org3,
        FABRIC_CA_SERVER_CA_CERTFILE=/etc/hyperledger/fabric-ca-server-config/ca.org3.example.com-cert.pem,
        FABRIC_CA_SERVER_CA_KEYFILE=/etc/hyperledger/fabric-ca-server-config/f65c26ee7c778d9c70058fd4fadd10a3676caa3c47d47bdc4f3e31574f96ac64_sk,
        FABRIC_CA_SERVER_TLS_ENABLED=true,
        FABRIC_CA_SERVER_TLS_CERTFILE=/etc/hyperledger/fabric-ca-server-config/ca.org3.example.com-cert.pem,
        FABRIC_CA_SERVER_TLS_KEYFILE=/etc/hyperledger/fabric-ca-server-config/f65c26ee7c778d9c70058fd4fadd10a3676caa3c47d47bdc4f3e31574f96ac64_sk,
      ]
    image: hyperledger/fabric-ca:1.4.9
    networks: [celloNet]
    ports: ["30006:7054"]
    volumes:
      [
        "../crypto-config/peerOrganizations/org3.example.com/ca/:/etc/hyperledger/fabric-ca-server-config",
      ]
  orderer-orderer:
    command: orderer
    container_name: orderer-orderer
    environment:
      [
        ORDERER_GENERAL_LOGLEVEL=DEBUG,
        ORDERER_GENERAL_LISTENADDRESS=0.0.0.0,
        ORDERER_GENERAL_LISTENPORT=30000,
        ORDERER_GENERAL_GENESISMETHOD=file,
        ORDERER_GENERAL_GENESISFILE=/var/hyperledger/orderer/orderer.genesis.block,
        ORDERER_GENERAL_LOCALMSPID=OrdererMSP,
        ORDERER_GENERAL_LOCALMSPDIR=/var/hyperledger/orderer/msp,
        ORDERER_GENERAL_TLS_ENABLED=true,
        ORDERER_GENERAL_TLS_PRIVATEKEY=/var/hyperledger/orderer/tls/server.key,
        ORDERER_GENERAL_TLS_CERTIFICATE=/var/hyperledger/orderer/tls/server.crt,
        "ORDERER_GENERAL_TLS_ROOTCAS=[/var/hyperledger/orderer/tls/ca.crt]",
        ORDERER_GENERAL_CLUSTER_CLIENTCERTIFICATE=/var/hyperledger/orderer/tls/server.crt,
        ORDERER_GENERAL_CLUSTER_CLIENTPRIVATEKEY=/var/hyperledger/orderer/tls/server.key,
        "ORDERER_GENERAL_CLUSTER_ROOTCAS=[/var/hyperledger/orderer/tls/ca.crt]",
      ]
    image: hyperledger/fabric-orderer:1.4.12
    networks: [celloNet]
    ports: ["30000:30000"]
    volumes:
      [
        "../crypto-config/ordererOrganizations/example.com/orderers/orderer.example.com/msp:/var/hyperledger/orderer/msp",
        "../crypto-config/ordererOrganizations/example.com/orderers/orderer.example.com/tls:/var/hyperledger/orderer/tls",
        "../channel-artifacts/genesis.block:/var/hyperledger/orderer/orderer.genesis.block",
      ]
  peer0.org2.example.com:
    command: peer node start
    container_name: peer0-org2
    environment:
      [
        CORE_PEER_ID=peer0.org2.example.com,
        CORE_PEER_LOCALMSPID=Org2MSP,
        "CORE_PEER_ADDRESS=peer0.org2.example.com:7051",
        CORE_VM_DOCKER_HOSTCONFIG_NETWORKMODE=deploy_celloNet,
        CORE_LOGGING_LEVEL=DEBUG,
        CORE_PEER_GOSSIP_USELEADERELECTION=true,
        CORE_PEER_GOSSIP_ORGLEADER=false,
        CORE_PEER_GOSSIP_SKIPHANDSHAKE=true,
        CORE_PEER_TLS_ENABLED=true,
        CORE_PEER_TLS_CERT_FILE=/etc/hyperledger/fabric/tls/server.crt,
        CORE_PEER_TLS_KEY_FILE=/etc/hyperledger/fabric/tls/server.key,
        CORE_PEER_TLS_ROOTCERT_FILE=/etc/hyperledger/fabric/tls/ca.crt,
        GODEBUG=netdns=go,
      ]
    image: hyperledger/fabric-peer:1.4.12
    networks: [celloNet]
    ports: ["30001:7051", "30002:7052"]
    volumes:
      [
        "/var/run/:/var/run/",
        "../crypto-config/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/msp:/etc/hyperledger/fabric/msp",
        "../crypto-config/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls:/etc/hyperledger/fabric/tls",
      ]
  peer0.org3.example.com:
    command: peer node start
    container_name: peer0-org3
    environment:
      [
        CORE_PEER_ID=peer0.org3.example.com,
        CORE_PEER_LOCALMSPID=Org3MSP,
        "CORE_PEER_ADDRESS=peer0.org3.example.com:7051",
        CORE_VM_DOCKER_HOSTCONFIG_NETWORKMODE=deploy_celloNet,
        CORE_LOGGING_LEVEL=DEBUG,
        CORE_PEER_GOSSIP_USELEADERELECTION=true,
        CORE_PEER_GOSSIP_ORGLEADER=false,
        CORE_PEER_GOSSIP_SKIPHANDSHAKE=true,
        CORE_PEER_TLS_ENABLED=true,
        CORE_PEER_TLS_CERT_FILE=/etc/hyperledger/fabric/tls/server.crt,
        CORE_PEER_TLS_KEY_FILE=/etc/hyperledger/fabric/tls/server.key,
        CORE_PEER_TLS_ROOTCERT_FILE=/etc/hyperledger/fabric/tls/ca.crt,
        GODEBUG=netdns=go,
      ]
    image: hyperledger/fabric-peer:1.4.12
    networks: [celloNet]
    ports: ["30004:7051", "30005:7052"]
    volumes:
      [
        "/var/run/:/var/run/",
        "../crypto-config/peerOrganizations/org3.example.com/peers/peer0.org3.example.com/msp:/etc/hyperledger/fabric/msp",
        "../crypto-config/peerOrganizations/org3.example.com/peers/peer0.org3.example.com/tls:/etc/hyperledger/fabric/tls",
      ]
version: "3.2"