hyperledger / indy-node-monitor

Apache License 2.0
13 stars 28 forks source link

Determine what information is available from validator-info to determine node compliance #24

Closed lohanspies closed 3 years ago

lohanspies commented 3 years ago

The objective is this exercise is to determine what information provided by the validator-info call can be used to measure compliance with the Sovrin Technical Policies

An example might be to determine if nodes have two Public IPs for node and client traffic respectively as per the technical policy.

Further analysis is required to determine what is already available vs what is missing. i.e. a Gap analysis, the outcome might be requested changes to the output provided by the validator-info call from indy-vdr

lohanspies commented 3 years ago

Node compliance info available with the current output of fetch_validator:

Things that we can potentially add to fetch_validator output:

Another option might be to force Stewards to provide the output of the compliance scripts once a month?

Any thoughts/suggestions? @WadeBarnes @swcurran

WadeBarnes commented 3 years ago

I think it's best if we update the validator-info for the nodes to return the additional information we want. i.e incorporate the collection of the data analyzed by the Technical Verification Script

WadeBarnes commented 3 years ago

By incorporating the collection of the additional data analyzed by the Technical Verification Script, it would be easy to implement continuous compliance monitoring.

lohanspies commented 3 years ago

@WadeBarnes it would be great if we can include the Technical Verification Script into validator_info.

Suggest we make this a privileged request the same as --status or detailed output from validator_info.

lohanspies commented 3 years ago

We will need to clone and clean the Technical Verification Script so that it can run without any user input and change the output format to JSON

WadeBarnes commented 3 years ago

@lohanspies, I recommend incorporating just the data collection elements of the Technical Verification Script into validator_info (of the node). The analysis portions would then be incorporated into the indy-node-monitor analysis (like detect_issues), and then summarized/reported in the status section of the output of indy-node-monitor. That way we're not imposing a specific network's requirements on anyone wishing to run a set of nodes.

By adding the additional details to the response from the node itself, the additional data would only be available to privileged requests as it is now.

lohanspies commented 3 years ago

Agree with this approach. Suggest we think about anything else we might want/need to add to the Technical Verification Script before starting this work.

Anything on top of mind from your side? Maybe a list of outstanding patches would be a great addition. That can feed into the security analysis of an Indy Network.

Food for thought - how can we get some security metrics out of this as well? Thinking out loud here and not a well thought through idea at this stage.

WadeBarnes commented 3 years ago

Agreed, I think it would be good to discuss further when we're designing the updates for validator_info in indy-node. If we're already in there updating it to collect additional data, we may as well try to be as complete as possible.

lohanspies commented 3 years ago

Exactly. We can continue the discussion during the SC Health Workstream call today. Thank you for the feedback so far.

lohanspies commented 3 years ago

A few discussion items for the Steward Council Health Workstream call today:

Ensure that the technical policies already enable the extraction of this type of information. ACLs on who is allowed to request, receive and store this information.

Transparency on blockchain state: (This information is available, just not via a public dashboard)

This approach does have some benefits:

Maybe a good idea is to present the approach to Stewards to get their input as well. Seems like there is still a need to help Stewards with detailed node monitoring.

@WadeBarnes update README to clearly indicate what analysis is available right now in indy-node-monitor; https://github.com/hyperledger/indy-node-monitor/issues/25

We will continue the discussion before actively starting to work on these proposed changes.

WadeBarnes commented 3 years ago

Summary of tickets created based on the conversations here:

WadeBarnes commented 3 years ago

@lohanspies, Please review the summarized tickets to see if they adequately address what's been discussed.

WadeBarnes commented 3 years ago

@lohanspies, For the following tickets, who is best equipped to start this investigation, and where is the relevant resource material?

lohanspies commented 3 years ago

@WadeBarnes reviewed the tickets and made comments on some of them. A key thing we might want to consider is to include information that can inform the node selection algorithm. Not sure if it belongs here but would be great to have a discussion with you on the topic.

WadeBarnes commented 3 years ago

@WadeBarnes reviewed the tickets and made comments on some of them. A key thing we might want to consider is to include information that can inform the node selection algorithm. Not sure if it belongs here but would be great to have a discussion with you on the topic.

@lohanspies, It would be best to treat that as a separate topic.

WadeBarnes commented 3 years ago

@lohanspies, Once we feel we've addressed the topics sufficiently in the spawned tickets I'd like to close this one as complete.

lohanspies commented 3 years ago

@WadeBarnes sure, can we please discuss this in the SC Health call tomorrow and ensure we are aligned on all tickets?

WadeBarnes commented 3 years ago

Absolutely

WadeBarnes commented 3 years ago

Reviewed on the Health Workstream Call and agreed the tickets cover the discussed items.