Add chaincode statistics API to peer

[dshuffma-ibm]

Description

Enhancement of peer APIs. We need statistics of network usage, this would include the following per chaincode:

• number of invokes
• timestamp of last invoke, utc unix ts in ms or sec
• timestamp of deploy, utc unix ts in ms or sec
• number of queries (that this peer has seen)
• timestamp of last query (that this peer has seen)
• the same path field sent in the deploy API (see example below)
• the same input field sent in the deploy API (see example below)

Also the statistic tracking can be disabled from the yaml file (for performance sensitive setups).

proposed json format

{
    chaincodes:{
        0645646alkdfjalsflkasjflajsfajsdlfjsafjlasjfasdflasjflajsfkasfsaf:{     //chaincode hash id
            invokes: 1123,
            last_invoke: 1464321600000,
            deployed: 1464321600000,
            queries: 10,                    //relative to this peer
            last_query: 1464321600000,          //relative to this peer
            path: "github.com/IBM-Blockchain/marbles-chaincode",
            input: {
                function:"init",
                args:["50"]
            }
        },
        35345alkdfjalsflkasjflajsfajsdlfjsafjlasjfasdflasjflajsfkasfsaf:{
            invokes: 123,
            last_invoke: 1464321600000,
            deployed: 1464321600000,
            queries: 234104,                //relative to this peer
            last_query: 1464321600000,          //relative to this peer
            path: "github.com/IBM-Blockchain/marbles-chaincode",
            input: {
                function:"init",
                args:["99"]
            }
        },
        456alkdfjalsflkasjflajsfajsdlfjsafjlasjfasdflasjflajsfkasfsaf:{
            invokes: 3,
            last_invoke: 1464321600000,
            deployed: 1464321600000,
            queries: 1024104,               //relative to this peer
            last_query: 1464321600000,          //relative to this peer
            path: "github.com/hyperledger/fabric/examples/chaincode/go/chaincode_example02",
            input: {
                function:"init",
                args:["a", "1000", "b", "2000"]
            }
        }
    }
}

[srderson]

This should be tracked in a a new column family in RocksDB with the ability to turn this feature on/off in core.yaml.

Adding a "help wanted" tag to see if anyone wants to take a shot at implementing this feature. Would be a bit more advanced, but nice as it's isolated from other features.

[kchristidis]

(If nobody offers for this, I'm here.)

[dshuffma-ibm]

I believe @angrbrd is interested in taking this up but needs a few days to figure out if she has the time. If she can't, I'll let you know!

[binhn]

What would the security of such data collection be, especially when the chaincode is confidential?

[GrapeBaBa]

i would like to try this one. Does someone can give a brief design or need do it from scratch?

[binhn]

@dshuffma-ibm @srderson @GrapeBaBa I would not implement this as part of peer, which has provided an event mechanism for any listener to get and accumulate such data (in general, analytics). Architecturally, we have presented and prototyped the concept of a warehouse, a component that may provide data for reporting and analytics without taxing the OLPT.

Since we have the client SDK, it can be added there via event listener.

[dshuffma-ibm]

@binh that's no good for our needs. This is an everyone problem isn't it? Everyone should be interested in details for their chaincode. At a minimal people would like to see/confirm what chaincode is running from the chaincode hash (ie show the go get url). As for performance, this would be an optional configuration. The performance hit for tracking a handful of integers and strings should be extremely minimal. (we are just counting things!) Actually your proposal to leverage the event listeners would be more wasteful wouldn't it? Communicating every single event out has to be worse then updating a local record.

[binhn]

@dshuffma-ibm It is not the performance that I am recommending alternative, but it is architectural as we wouldn't want to keep adding things to the peer such as keeping track of stats. Moreover, the ask data would also cause security concerns.

[dshuffma-ibm]

the input.args field is potentially compromising, since it would expose the init parameters. one alternative to this particular field would be to limit the access to different members/enrollIDs based on group. We could ditch the field altogether, its actually not needed for us.

the other fields are pretty bland and are harmless in my opinion. Someone might be hesitant to show their go get url but, this api wouldn't be public anyway. Only registered enrollIDs should be able to see it. This url is really useful information for anyone using hyperledger.

i would stand by my prev statement that everyone wants to see these stats, so why make it an external feature.

[binhn]

@dshuffma-ibm you might want to look at the pattern here https://github.com/ethereum/wiki/wiki/Network-Status where the https://github.com/cubedro/eth-net-intelligence-api is like an event listener.

[dshuffma-ibm]

fyi we have an intern assigned to this task, but he's got other projects too, so I don't think much headway has been made yet.

If we are not interested in adding this feature to the fabric, then we would be forced to explore client sdk per network (like you suggest)...

I'm still in favor of having a peer track its own stats.

[ajpauwels]

I have completed this, here is a link to the fork: https://github.com/ajpauwels/fabric/tree/addstatsapi

It tracks successful vs. failed invokes, but this only reflects whether the peer was able to broadcast a consensus message for the invoke. It is not possible to synchronously retrieve whether or not the invoke itself was successful with the current state of the fabric. This should be fine for now, we can revisit at a later date. Failed invokes essentially represent a networking issue.

[christo4ferris]

@ajpauwels why not submit a PR so that it can be reviewed? @dshuffma-ibm

[ajpauwels]

@christo4ferris Happy to but want to make sure this is useful and fits the architecture, especially re: discussion above.

I've found it useful so far in tracking network health per-peer to quickly identify failing nodes. Since the stats are tracked locally based on success in sending broadcast messages, if a node suddenly starts failing to communicate with the other peers it can be quickly identified by a discrepancy in the stats. I think this is more useful for identifying network health than an events-based system which would be network-wide and could only track globally whether an invoke failed or succeeded. Ideally, you would have both in order to also track chaincode-related errors once the network is running.

I've also been spinning up development networks both locally (node app on one VM and fabric on another) and remotely on vlaunch and it's been very handy as a quick REST call to make sure the dev net is up and communicating properly.

Architecturally, I wouldn't compare local peer stats tracking to an event-based stats tracking as direct competitors (at least, in the way that events are implemented right now). Events are strictly a global success/failure per transaction and don't give any details on whether certain peers succeeded or not. Rather, I'd say the two could work together to give a complete view of network health.

[bravera]

Does this maintain a history of stats records or is there just a single record per chaincode which is getting updated? If its the latter, can the record be reconstructed by replaying the blockchain transactions or are there pieces of information being stored (and hence lost on the next update) that are not derivable from elsewhere on the chain? Just thinking about whether its possible for the stats records to become out of step with reality in the event of a failure, or even a scheduled outage due to CI maybe, and valuable information being lost.

[ajpauwels]

@bravera This maintains a single record per chaincode that is actively updated. This information persists as long as the peer's RocksDB database isn't cleared out.

The record cannot be reconstructed by replaying transactions since failed transactions are not stored on the blockchain.

Not sure exactly what you mean by last sentence. Do you mind clarifying?