Working code in docker dev fails on authentication with kube cluster version

[rddill-IBM]

I have nodejs app supporting a dispute resolution network. The code runs successfully in the local, docker-based dev version, but fails consistently with authentication errors for users. I am running v0.16.2. The process I'm following (IBM Cloud, Free Kube Cluster) is:

• ./delete_all.sh
• ./create_all.sh
• go to playground, export PeerAdmin card
• ./connection-profile/update_card.sh with valid parameters
• composer card import -f ~/yada yada
• composer card list (shows correctly imported card). Inspecting data shows that the card points correctly to my Kube cluster
• change over to my network/dist folder and
• composer runtime install --card PeerAdmin@hlfv1 --businessNetworkName zerotoblockchain-network
• composer network start -c PeerAdmin@hlfv1 -A admin -S adminpw -a zerotoblockchain-network.bna --file admin.card
• composer card import --file admin.card
• composer card list (yup, card is present)
• composer network ping -c admin@zerotoblockchain-network (MUST do this step, otherwise nodejs fails with admin id not enrolled.

Transition to nodejs app. one function in the app preloads the (demo) environment with Assets and Members. The code used to implement member creation follows.

    adminConnection.connect(config.composer.adminCard)
    .then(() => {
        // a businessNetworkConnection is required to add members
        businessNetworkConnection = new BusinessNetworkConnection();
        // connection prior to V0.15
        // return businessNetworkConnection.connect(config.composer.connectionProfile, config.composer.network, config.composer.adminID, config.composer.adminPW)
        // connection in v0.15
        return businessNetworkConnection.connect(config.composer.adminCard)
        .then(() => {
            // a factory is required to build the member object
            factory = businessNetworkConnection.getBusinessNetwork().getFactory();
            //iterate through the list of members in the memberList.json file and
            // first add the member to the network, then create an identity for
            // them. This generates the memberList.txt file later used for
            // retrieving member secrets.
            for (let each in startupFile.members)
                {(function(_idx, _arr)
                    {
                    // the participant registry is where member information is first stored
                    // there are individual registries for each type of participant, or member.
                    // In our case, that is Buyer, Seller, Provider, Shipper, FinanceCo
                    return businessNetworkConnection.getParticipantRegistry(config.composer.NS+'.'+_arr[_idx].type)
                    .then(function(participantRegistry){
                        return participantRegistry.get(_arr[_idx].id)
                        .then((_res) => {
                            console.log('['+_idx+'] member with id: '+_arr[_idx].id+' already exists in Registry '+config.composer.NS+'.'+_arr[_idx].type);
                            svc.m_connection.sendUTF('['+_idx+'] member with id: '+_arr[_idx].id+' already exists in Registry '+config.composer.NS+'.'+_arr[_idx].type);
                        })
                        .catch((error) => {
                            participant = factory.newResource(config.composer.NS, _arr[_idx].type, _arr[_idx].id);
                            participant.companyName = _arr[_idx].companyName;
                            participantRegistry.add(participant)
                            .then(() => {
                                console.log('['+_idx+'] '+_arr[_idx].companyName+' successfully added');
                                svc.m_connection.sendUTF('['+_idx+'] '+_arr[_idx].companyName+' successfully added');
                            })
                            .then(() => {
                                // an identity is required before a member can take action in the network.
                                // V0.14
                                // return businessNetworkConnection.issueIdentity(config.composer.NS+'.'+_arr[_idx].type+'#'+_arr[_idx].id, _arr[_idx].pw)
                                // V0.15
                                console.log('issuing identity for: '+config.composer.NS+'.'+_arr[_idx].type+'#'+_arr[_idx].id);
                                return businessNetworkConnection.issueIdentity(config.composer.NS+'.'+_arr[_idx].type+'#'+_arr[_idx].id, _arr[_idx].id)
                                .then((result) => {
                                    console.log('_arr[_idx].id: '+_arr[_idx].id);
                                    console.log('result.userID: '+result.userID);
                                    let _mem = _arr[_idx];
                                    _mem.secret = result.userSecret;
                                    _mem.userID = result.userID;
                                    memberTable.push(_mem);
                                    // svc.saveMemberTable(memberTable);
                                    let _meta = {};
                                    for (each in config.composer.metaData)
                                    {(function(_idx, _obj) {_meta[_idx] = _obj[_idx]; })(each, config.composer.metaData); }
                                    _meta.businessNetwork = config.composer.network;
                                    _meta.userName = result.userID;
                                    _meta.enrollmentSecret = result.userSecret;
                                    let tempCard = new hlc_idCard(_meta, config.connectionProfile);
                                    return adminConnection.importCard(result.userID, tempCard)
                                        .then ((_res) => { 
                                            if (_res) {console.log('card updated');} else {console.log('card imported');}
                                })
                                .catch((error) => {
                                    console.error('create id for '+_arr[_idx].id+'failed. ',error.message);
                                });
                            })
                        .catch((error) => {console.log(_arr[_idx].companyName+' add failed',error.message);});
                        });
                    })
                .catch((error) => {console.log('error with getParticipantRegistry', error.message);});
                })(each, startupFile.members);

Later in the application, I then use the id's for the members to connect to the business network and perform relevant functions. That code works 100% of the time in the Docker developer instance. In the Kubernetes environment, my app is failing 100% on the Business Network Connect function using the id's which were created in this process. the error is:

Error trying login and get user Context. 
Error: Error trying to enroll user or load channel configuration. 
Error: Enrollment failed with errors [[{"code":400,"message":"Authorization failure"}]]

I thought that perhaps I was now required to invoke a bindIdentity after the issueIdentity, but that fails with

Failed to add object with ID '808d50449fa2f2cdeaf40016aab49010261365b57ba106410616f7dbc9594d94' as the object already exists

in hlf_connection/client.js

I'm stumped. been working on this for 2 days trying to tease out where the problem lies. I'm using the following for a connection profile (based on Docker dev enviroment, updated with correct ip address for Kube cluster). I am unsureif in your Kube cluster you're still using composerchannel in the Kube cluster. the ca element used to have a name element, but ca.org1.example.com always came up with a 404, so I removed that from the profile.

    "connectionProfile":
    {
        "name": "hlfv1",
        "type": "hlfv1",
        "orderers": [ { "url": "grpc://173.193.99.29:31010" } ],
        "ca": { "url": "http://173.193.99.29:30054" },
        "peers": [ { "requestURL": "grpc://173.193.99.29:30110", "eventURL": "grpc://173.193.99.29:30111" } ],
        "keyValStore": "/.composer-credentials",
        "channel": "composerchannel",
        "mspID": "Org1MSP",
        "timeout": 300 
    }

[mrshah-at-ibm]

@davidkel can we get your help here please.

[davidkel]

I am guessing that you get that error message when you invoke the issueIdentity api ? That error comes back from the fabric-ca server when you try to use an identity that isn't recognised by the ca or isn't allowed to issue new identities. The logs in the ca server would provide more detail as to the exact problem I would hope. Also run your application with sdk debug turned on export DEBUG=composer:* it will generate a log file in the composer-logs directory which may also provide some useful info.

[rddill-IBM]

David, When I run my app in the Kubernetes environment, it fails when any user other than admin attempts to connect to the Business Network. All of the participants in the network fail with the

Error trying login and get user Context. 
Error: Error trying to enroll user or load channel configuration. 
Error: Enrollment failed with errors [[{"code":400,"message":"Authorization failure"}]]

error sequence. I attempted to resolve this error by adding code to invoke bindIdentity after a successful issueIdentity request. That sequence, shown in the preceding code block, fails with the 'object already exists' error message. The problem I'm really trying to solve is the first error message.

[davidkel]

bindIdentity is not what you need. That error comes from the fabric-ca-server. The logs of the fabric-ca-server might indicate why it is giving an authorisation failure. The error is not too helpful, it could be the result of trying to issue an identity using an identity which is not authorised or it could be that you are trying to enroll a user that has already been enrolled for example (that problem could occur only if the fabric-ca-server has got a certain fix applied and I don't know what level of fabric that fix is in). I would also suggest getting the logs from your application as well as described in my previous append as it would show the flow of requests made to the ca server.

[rddill-IBM]

david, thanks. It turns out that the CA name and Channel Name in the Kubernetes configuration are different from what was used in the docker version. I've updated my install process to automatically extract the correct profile information from the admin card and use that data (specifically the content from connection.json) as my template when creating new users.