kuujo
commented
11 years ago Hmm... this could also cause problems if a user deploys a module with the -ha option, that module deploys a network through a ViaCluster, and then the original Vert.x instance dies. That would result in any network components deployed in that instance being reassigned to another node plus the originally deployed module being reassigned to another node, which would in turn cause it to be deployed twice. This can result in some major conflicts because of event bus addressing - it would cause lots of unpredictable behavior (messages would be distributed between both deployed networks, coordinators wouldn't be able to reliably monitor component instances, connections between components wouldn't work properly because component instances would be subscribing to output from components in the other network, consistent hashing would not work properly because there would be double the connections, etc).
If the new ClusterManager API can't be co-opted by Vertigo then the documentation will have to warn against deploying modules using the -ha option. Alternatively, mechanisms could be added to ensure a network can't be deployed twice. Ideally, we don't want to link the ability of a network to run to the status of the deploying module or verticle, so we don't want to kill all network components if the deploying verticle's Vert.x instance dies. The intended behavior is to leave portions of the network deployed while components that were contained in the failed Vert.x instance are reassigned and redeployed.
Imagine the following scenario:
- Module is deployed with the
-haoption - Module deploys a Vertigo network using a
ViaClusterinstance - Module's Vert.x instance fails
- Module is deployed in another Vert.x instance
- Module attempts to deploy the same Vertigo network
- Network deployment fails due to the fact the network is already running
In this scenario, how does the module code know that the deployment failed because the network is already running? Maybe we would have to use something like a DeployFailure which would be similar to the new event bus ReplyFailure. Still, I don't think it's a very elegant solution. The new API supports JSON-based networks, so adding a net.kuujo~vertigo-network~0.1 module specifically designed for deploying networks from JSON would allow us to control these types of errors.
This HA thing may get complicated for a while.
ramukima
Hmmm.... that's a good question! I hadn't noticed this feature.
So, I think this will make Via sort of obsolete. Although, one of the features of Via is custom schedulers so that deployments can be assigned to specific nodes. Maybe I'll have to talk to Tim Fox and find out exactly what his plans for HA are. I think what will change depends on what is going into Vert.x.
Also, because users are deploying so many modules and verticles within a network, Vertigo needs to be able to have a scheduler spread those modules and verticles across multiple Vert.x instances when they're deployed. The
-haoption seems to deploy the module within a new Vert.x instance and only if the module dies does it get reassigned. This is great for deploying modules from the command line, but Vertigo will certainly require some different behavior.It would be nice if there were an API that at least allows a module to be deployed to a random node in the cluster.
I'll have to dig into the code to see what I can accomplish with the underlying implementation. Perhaps the Vert.x HA implementation can be used to deploy modules and verticles in Vertigo, and perhaps a scheduler can still be implemented on top of it. I'd love to use as much of the core Vert.x cluster manager support as possible rather than building anything on top of it. Tim has said that he wanted a cluster manager in Vert.x, so I knew this would happen. The faster it gets into core the faster we can offload that type of complicated work to a core API. Maybe I can influence some APIs that would allow something like a scheduler to be implemented on top of the Vert.x implementation.