`config-tool` updates

[mathieucarbou]

config-tool support for multiple -connect-to parameters for get|set|unset|import|diagnostic|activate

For activate:

Supporting multiple -connect-to parameters is useful in the case of a repair when force-pushing and force-activating a topology to some nodes, instead of doing node by node.

For get|set|unset:

Supporting multiple -connect-to parameters allows the caller to pass all the known nodes of the cluster, and config-tool will pick one to fetch the topology. These commands can be run with some nodes down so supporting multiple values allows to pass all known pods in the CLI in a cloud env.

For import:

Same for activate: allows tp import a cluster config to several nodes at once

For diagnostic:

Same as get|set : diag command can be run over a cluster where some nodes are down, so it needs to support a complete set of pods and will detect the ones reachable

All these use cases are not only for cloud but for normal deployments, but they are really valuable on a cloud env.

config-tool.sh diagnostic -connect-to localhost -output-format json

Enables an easier processing of diagnotic output.

For Kube Operator, these fields are relevant:

• manualInterventionRequired
• readyForTopologyChange

Example for a node in diagnostic mode

• manualInterventionRequired: false
• readyForTopologyChange: false

{
  "configChangeCommittedAll" : false,
  "configChangeCommittedOnline" : false,
  "configChangeInProgress" : false,
  "configChangePartiallyCommitted" : false,
  "configChangePartiallyPrepared" : false,
  "configChangePartiallyRolledBack" : false,
  "configChangePreparedAll" : false,
  "configChangePreparedOnline" : false,
  "configDiscoveryFailed" : false,
  "configDiscoveryFailure" : null,
  "configInconsistent" : false,
  "configLocked" : false,
  "configLockedToken" : null,
  "configPartitioned" : false,
  "manualInterventionRequired" : false,
  "nodeStates" : {
    "DIAGNOSTIC" : 1
  },
  "nodes" : 1,
  "nodesInActiveMode" : 0,
  "nodesInConfigMode" : 1,
  "nodesInRepairMode" : 0,
  "nodesOnline" : 1,
  "nodesRequiringRestart" : 0,
  "nodesUnreachable" : 0,
  "readyForTopologyChange" : false,
  "stripes" : 1
}

Example for an activated node

• manualInterventionRequired: false
• readyForTopologyChange: true

{
  "configChangeCommittedAll" : true,
  "configChangeCommittedOnline" : false,
  "configChangeInProgress" : false,
  "configChangePartiallyCommitted" : false,
  "configChangePartiallyPrepared" : false,
  "configChangePartiallyRolledBack" : false,
  "configChangePreparedAll" : false,
  "configChangePreparedOnline" : false,
  "configDiscoveryFailed" : false,
  "configDiscoveryFailure" : null,
  "configInconsistent" : false,
  "configLocked" : false,
  "configLockedToken" : null,
  "configPartitioned" : false,
  "manualInterventionRequired" : false,
  "nodeStates" : {
    "ACTIVE" : 1
  },
  "nodes" : 1,
  "nodesInActiveMode" : 1,
  "nodesInConfigMode" : 0,
  "nodesInRepairMode" : 0,
  "nodesOnline" : 1,
  "nodesRequiringRestart" : 0,
  "nodesUnreachable" : 0,
  "readyForTopologyChange" : true,
  "stripes" : 1
}

Example for a node in repair mode

• manualInterventionRequired: false
• readyForTopologyChange: false

{
  "configChangeCommittedAll" : true,
  "configChangeCommittedOnline" : false,
  "configChangeInProgress" : false,
  "configChangePartiallyCommitted" : false,
  "configChangePartiallyPrepared" : false,
  "configChangePartiallyRolledBack" : false,
  "configChangePreparedAll" : false,
  "configChangePreparedOnline" : false,
  "configDiscoveryFailed" : false,
  "configDiscoveryFailure" : null,
  "configInconsistent" : false,
  "configLocked" : false,
  "configLockedToken" : null,
  "configPartitioned" : false,
  "manualInterventionRequired" : false,
  "nodeStates" : {
    "DIAGNOSTIC" : 1
  },
  "nodes" : 1,
  "nodesInActiveMode" : 0,
  "nodesInConfigMode" : 0,
  "nodesInRepairMode" : 1,
  "nodesOnline" : 1,
  "nodesRequiringRestart" : 0,
  "nodesUnreachable" : 0,
  "readyForTopologyChange" : false,
  "stripes" : 1
}

Example when restart is required

• manualInterventionRequired: true
• readyForTopologyChange: false

{
  "configChangeCommittedAll" : true,
  "configChangeCommittedOnline" : false,
  "configChangeInProgress" : false,
  "configChangePartiallyCommitted" : false,
  "configChangePartiallyPrepared" : false,
  "configChangePartiallyRolledBack" : false,
  "configChangePreparedAll" : false,
  "configChangePreparedOnline" : false,
  "configDiscoveryFailed" : false,
  "configDiscoveryFailure" : null,
  "configInconsistent" : false,
  "configLocked" : false,
  "configLockedToken" : null,
  "configPartitioned" : false,
  "manualInterventionRequired" : true,
  "nodeStates" : {
    "PASSIVE" : 1,
    "ACTIVE" : 1
  },
  "nodes" : 2,
  "nodesInActiveMode" : 2,
  "nodesInConfigMode" : 0,
  "nodesInRepairMode" : 0,
  "nodesOnline" : 2,
  "nodesRequiringRestart" : 1,
  "nodesUnreachable" : 0,
  "readyForTopologyChange" : false,
  "stripes" : 1
}

Examples when config is locked

• manualInterventionRequired: false
• readyForTopologyChange: false

{
  "configChangeCommittedAll" : true,
  "configChangeCommittedOnline" : false,
  "configChangeInProgress" : false,
  "configChangePartiallyCommitted" : false,
  "configChangePartiallyPrepared" : false,
  "configChangePartiallyRolledBack" : false,
  "configChangePreparedAll" : false,
  "configChangePreparedOnline" : false,
  "configDiscoveryFailed" : false,
  "configDiscoveryFailure" : null,
  "configInconsistent" : false,
  "configLocked" : true,
  "configLockedToken" : "some-uuid",
  "configPartitioned" : false,
  "manualInterventionRequired" : false,
  "nodeStates" : {
    "ACTIVE" : 1
  },
  "nodes" : 1,
  "nodesInActiveMode" : 1,
  "nodesInConfigMode" : 0,
  "nodesInRepairMode" : 0,
  "nodesOnline" : 1,
  "nodesRequiringRestart" : 0,
  "nodesUnreachable" : 0,
  "readyForTopologyChange" : false,
  "stripes" : 1
}

Example commit failure:

{
  "configChangeCommittedAll" : false,
  "configChangeCommittedOnline" : false,
  "configChangeInProgress" : false,
  "configChangePartiallyCommitted" : false,
  "configChangePartiallyPrepared" : false,
  "configChangePartiallyRolledBack" : false,
  "configChangePreparedAll" : true,
  "configChangePreparedOnline" : false,
  "configDiscoveryFailed" : false,
  "configDiscoveryFailure" : null,
  "configInconsistent" : false,
  "configLocked" : false,
  "configLockedToken" : null,
  "configPartitioned" : false,
  "manualInterventionRequired" : true,
  "nodeStates" : {
    "ACTIVE" : 1
  },
  "nodes" : 1,
  "nodesInActiveMode" : 1,
  "nodesInConfigMode" : 0,
  "nodesInRepairMode" : 0,
  "nodesOnline" : 1,
  "nodesRequiringRestart" : 0,
  "nodesUnreachable" : 0,
  "readyForTopologyChange" : false,
  "stripes" : 1
}