encountered problem using docker-in-docker for Hyperledger Fabric

[sxguan]

The error coming from the inner(Docker-in-Docker) container. The outer container image is ubuntu official image 'ubuntu:20.04' (launched with Docker + Sysbox). And the inner container image is Hyperledger Fabric official image 'hyperledger/fabric-tools:latest' (Linux 5.4.0-58-generic #64-Ubuntu).

bash-5.0# peer lifecycle chaincode install sacc.tar.gz
Error: chaincode install failed with status: 500 - failed to invoke backing implementation of 'InstallChaincode': could not build chaincode: docker build failed: docker image build failed: docker build failed: Error returned from build: 1 "failed to initialize build cache at /home/chaincode/.cache/go-build: mkdir /home/chaincode/.cache: permission denied"

The inner container's GOCACHE set asGOCACHE="/root/.cache/go-build"

[ctalledo]

Hi @sxguan,

Can you provide the docker run commands you used for both the outer container and the inner container?

Thanks!

[rodnymolina]

@sxguan, aside from what Cesar mentioned above, please also provide the Hyperledger specific instructions that you are executing and any required yaml config file. Please keep in mind that we are not familiarized with Hyperledger.

As you can see, I'm obtaining a different error than you, probably due to a lack of Hyperledger configuration ...

/go # peer lifecycle chaincode install sacc.tar.gz
Error: failed to retrieve endorser client for install: endorser client failed to connect to 0.0.0.0:7051: failed to create new connection: connection error: desc = "transport: error while dialing: dial tcp 0.0.0.0:7051: connect: connection refused"
Usage:
  peer lifecycle chaincode install [flags]

[sxguan]

Hi @ctalledo @rodnymolina, Thanks for your reply. The outer container command is docker run --runtime=sysbox-runc -itd -p 5901:5901 ubuntu:20.04 bash. The inner container was up by docker-compose.yaml as following:

version: '2'

volumes:
  orderer.yishu.com:
  peer0.org1.yishu.com:

networks:
  yishu:

services:

  orderer.yishu.com:
    container_name: orderer.yishu.com
    image: hyperledger/fabric-orderer:latest
    environment:
      - FABRIC_LOGGING_SPEC=INFO
      - ORDERER_GENERAL_LISTENADDRESS=0.0.0.0
      - ORDERER_GENERAL_LISTENPORT=7050
      - ORDERER_GENERAL_GENESISMETHOD=file
      - ORDERER_GENERAL_GENESISFILE=/var/hyperledger/orderer/orderer.genesis.block
      - ORDERER_GENERAL_LOCALMSPID=OrdererMSP
      - ORDERER_GENERAL_LOCALMSPDIR=/var/hyperledger/orderer/msp
      # enabled TLS
      - ORDERER_GENERAL_TLS_ENABLED=true
      - ORDERER_GENERAL_TLS_PRIVATEKEY=/var/hyperledger/orderer/tls/server.key
      - ORDERER_GENERAL_TLS_CERTIFICATE=/var/hyperledger/orderer/tls/server.crt
      - ORDERER_GENERAL_TLS_ROOTCAS=[/var/hyperledger/orderer/tls/ca.crt]
      - ORDERER_KAFKA_TOPIC_REPLICATIONFACTOR=1
      - ORDERER_KAFKA_VERBOSE=true
      - ORDERER_GENERAL_CLUSTER_CLIENTCERTIFICATE=/var/hyperledger/orderer/tls/server.crt
      - ORDERER_GENERAL_CLUSTER_CLIENTPRIVATEKEY=/var/hyperledger/orderer/tls/server.key
      - ORDERER_GENERAL_CLUSTER_ROOTCAS=[/var/hyperledger/orderer/tls/ca.crt]
    working_dir: /opt/gopath/src/github.com/hyperledger/fabric
    command: orderer
    volumes:
        - ./channel-artifacts/genesis.block:/var/hyperledger/orderer/orderer.genesis.block
        - ./crypto-config/ordererOrganizations/yishu.com/orderers/orderer.yishu.com/msp:/var/hyperledger/orderer/msp
        - ./crypto-config/ordererOrganizations/yishu.com/orderers/orderer.yishu.com/tls/:/var/hyperledger/orderer/tls
        - orderer.yishu.com:/var/hyperledger/production/orderer
    ports:
      - 7050:7050
    networks:
      - yishu

  peer0.orggo.yishu.com:
    container_name: peer0.org1.yishu.com
    image: hyperledger/fabric-peer:latest
    environment:
      - CORE_VM_ENDPOINT=unix:///host/var/run/docker.sock
      - CORE_VM_DOCKER_HOSTCONFIG_NETWORKMODE=solotest_yishu
      - FABRIC_LOGGING_SPEC=INFO
      - CORE_PEER_TLS_ENABLED=true
      - CORE_PEER_GOSSIP_USELEADERELECTION=true
      - CORE_PEER_GOSSIP_ORGLEADER=false
      - CORE_PEER_PROFILE_ENABLED=true
      - CORE_PEER_TLS_CERT_FILE=/etc/hyperledger/fabric/tls/server.crt
      - CORE_PEER_TLS_KEY_FILE=/etc/hyperledger/fabric/tls/server.key
      - CORE_PEER_TLS_ROOTCERT_FILE=/etc/hyperledger/fabric/tls/ca.crt
      - CORE_PEER_ID=peer0.org1.yishu.com
      - CORE_PEER_ADDRESS=peer0.org1.yishu.com:7051
      - CORE_PEER_LISTENADDRESS=0.0.0.0:7051
      - CORE_PEER_CHAINCODEADDRESS=peer0.org1.yishu.com:7052
      - CORE_PEER_CHAINCODELISTENADDRESS=0.0.0.0:7052
      - CORE_PEER_GOSSIP_BOOTSTRAP=peer0.org1.yishu.com:7051
      - CORE_PEER_GOSSIP_EXTERNALENDPOINT=peer0.org1.yishu.com:7051
      - CORE_PEER_LOCALMSPID=Org1MSP
    volumes:
        - /var/run/:/host/var/run/
        - ./crypto-config/peerOrganizations/org1.yishu.com/peers/peer0.org1.yishu.com/msp:/etc/hyperledger/fabric/msp
        - ./crypto-config/peerOrganizations/org1.yishu.com/peers/peer0.org1.yishu.com/tls:/etc/hyperledger/fabric/tls
        - peer0.org1.yishu.com:/var/hyperledger/production
    working_dir: /opt/gopath/src/github.com/hyperledger/fabric/peer
    command: peer node start
    ports:
      - 7051:7051
      - 7053:7053
    networks:
      - yishu

  cli:

    container_name: cli
    image: hyperledger/fabric-tools:latest
    tty: true
    environment:
      - GOPATH=/opt/gopath
      - CORE_VM_ENDPOINT=unix:///host/var/run/docker.sock
      - FABRIC_LOGGING_SPEC=INFO
      - GODEBUG=netdns=go
      - CORE_PEER_ID=cli
      - CORE_PEER_ADDRESS=peer0.org1.yishu.com:7051
      - CORE_PEER_LOCALMSPID=Org1MSP
      - CORE_PEER_TLS_ENABLED=true
      - CORE_PEER_TLS_CERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.yishu.com/peers/peer0.org1.yishu.com/tls/server.crt
      - CORE_PEER_TLS_KEY_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.yishu.com/peers/peer0.org1.yishu.com/tls/server.key
      - CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.yishu.com/peers/peer0.org1.yishu.com/tls/ca.crt
      - CORE_PEER_MSPCONFIGPATH=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.yishu.com/users/Admin@org1.yishu.com/msp
    working_dir: /opt/gopath/src/github.com/hyperledger/fabric/peer 
    volumes:
      - /var/run/:/host/var/run/
      - ./chaincode:/opt/gopath/src/github.com/chaincode
      - ./crypto-config:/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/
      - ./channel-artifacts:/opt/gopath/src/github.com/hyperledger/fabric/peer/channel-artifacts
    networks:
      - yishu

The error was coming from container cli.

[sxguan]

Hi @ctalledo @rodnymolina , I have pushed configuration files. And commands as following: docker run --runtime=sysbox-runc -it ubuntu:20.04 git clone https://github.com/sxguan/solotest.git cd solotest docker-compose up -d docker exec -it cli bash peer lifecycle chaincode package sacc.tar.gz \ --path github.com/chaincode/sacc/ \ --label sacc_1 peer lifecycle chaincode install sacc.tar.gz

[rodnymolina]

@sxguan, thanks for your detailed instructions, it helps a lot!

Question. Did you build Sysbox yourself or did you install it making use of our latest deb package installer? I'm asking because I haven't reproduced the problem that you described, looks like it's working for me (see below). If you are relying on the deb package, please build Sysbox and try again (we have fixed multiple issues over the last few months).

Notice that I've made minor changes to your list of instructions to allow these ones to work (see them below). These differences are not relevant to the problem that you observed, so we can obviate them, I'm pasting them below in case you want to follow exactly what I've done.

• Minor setup instruction differences

$ docker run --runtime=sysbox-runc -it nestybox/ubuntu-focal-systemd-docker
...
root@440bdde2076b:/home/admin/solotest# git diff
diff --git a/docker-compose.yaml b/docker-compose.yaml
index e8687f8..6122285 100644
--- a/docker-compose.yaml
+++ b/docker-compose.yaml
@@ -11,7 +11,7 @@ services:

   orderer.yishu.com:
     container_name: orderer.yishu.com
-    image: hyperledger/fabric-orderer:latest
+    image: hyperledger/fabric-orderer:2.3
     environment:
       - FABRIC_LOGGING_SPEC=INFO
       - ORDERER_GENERAL_LISTENADDRESS=0.0.0.0
@@ -44,7 +44,7 @@ services:

   peer0.orggo.yishu.com:
     container_name: peer0.org1.yishu.com
-    image: hyperledger/fabric-peer:latest
+    image: hyperledger/fabric-peer:2.3
     environment:
       - CORE_VM_ENDPOINT=unix:///host/var/run/docker.sock
       - CORE_VM_DOCKER_HOSTCONFIG_NETWORKMODE=solotest_yishu
@@ -80,7 +80,7 @@ services:
   cli:

     container_name: cli
-    image: hyperledger/fabric-tools:latest
+    image: hyperledger/fabric-tools:2.3
     tty: true
     environment:
       - GOPATH=/opt/gopath
root@440bdde2076b:/home/admin/solotest#

• Reproduction attempt (seems to be working fine)

admin@440bdde2076b:~/solotest$ docker exec -it cli bash

bash-5.0# peer lifecycle chaincode package sacc.tar.gz --path github.com/chaincode/sacc/ --label sacc_1
bash-5.0#

bash-5.0# peer lifecycle chaincode install sacc.tar.gz
2021-01-05 19:00:22.178 UTC [cli.lifecycle.chaincode] submitInstallProposal -> INFO 001 Installed remotely: response:<status:200 payload:"\nGsacc_1:eff8444d13e6eed1462aa5fdaca27e5ec7073a7d2cd887b1eef2d9fbd01737f0\022\006sacc_1" >
2021-01-05 19:00:22.178 UTC [cli.lifecycle.chaincode] submitInstallProposal -> INFO 002 Chaincode code package identifier: sacc_1:eff8444d13e6eed1462aa5fdaca27e5ec7073a7d2cd887b1eef2d9fbd01737f0
bash-5.0#

[sxguan]

@rodnymolina ,thanks for your reply and work! I tried it again and it worked successfully.Maybe something wrong with vncserver.Cause I used vncserver to do this work before. All in all, I really appreciate your help!

[rodnymolina]

@sxguan, good to hear that. Please let us know if you run into any other issue.

One quick favor. Could you please provide a brief explanation of what's the use-case that you have in mind for Sysbox? Is this Hyperledger Fabric setup a testing scenario for your personal use, or are you planning to run this as a production environment of some sort? I'm just curious, that's all.

Thanks.

[sxguan]

@rodnymolina The project I am doing is a teaching platform which is embedded with a virtual machine.The virtual machine is realized by docker(with vncserver) on the server.By this way, students can learn and run Hyperledger Fabric on the teaching platform.So sysbox is crucial.Thank you and your team again for such a good product.

[rodnymolina]

Everything appears to be working fine. Closing issue now.