jboss as7 deploy/undeploy returns before completed

[aledsage]

When using jboss AS7 entity (or tomcat or as6), when you deploy or undeploy an artifact the effector returns as soon as the file has been copied. The web-app will not be available until some time after that, or the deployment may subsequently fail in the app-server without the brooklyn entity reporting this at all.

This is a particular problem for things like scaling a cluster. The load-balancer (e.g. nginx) will be updated to route requests to the new app-server, but this does not guarantee that the app-server has installed the required initial wars. It looks like this is a cause of failed http requests when scaling out a cluster.

---

The start-up should block until the initial wars have been deployed. And the deploy/undeploy effectors should block until the app-server really has finished the operation for the web-app.

[aledsage]

For AS7, I suggest we use the jboss-cli.sh to deploy and undeploy, rather than just adding/removing from the deployments dir. This is the recommendation in the admin guide:

https://docs.jboss.org/author/display/AS7/Admin+Guide#AdminGuide-DeploymentCommands
Users, particularly those running production systems, are encouraged to use the JBoss AS 7 
management APIs to upload and deploy deployment content instead of relying on the deployment 
scanner subsystem that periodically scans this directory.

[aledsage]

This is also an issue with Tomcat and AS6, but the solution for each of those will presumably look quite different.

[ahgittin]

jboss to use CLI is a good improvement. i wonder if it blocks until complete...

as for the races, on start/deploy:

• could have an option for a URL test in config (e.g. flag checkUrlPath: "/foo", checkUrlPathRegex: ".Hello Foo." ... or configuration method checkUrlPathContainsRegex("/foo", "Hello Foo")

And on stop:

• ensure controller (nginx) is updated before the server is shutdown by policy (eg emitting a STOPPING event and blocking until it is completely handled, or using a delay -- delay might be the pragmatic option since knowing when events are completely handled could be expensive)

and either:

• configure nginx/controller to retry if the server crashes (if possible)
• delay long enough before appserver shutdown that nginx will have reloaded and sessions to old server completed
• delay appserver shutdown until old server has no requests

again delay might be the pragmatic option