embedded web container does not shutdown properly

[glassfishrobot]

During embedded testing I noticed that embedded web container is not shutting down properly. Basically WebContainer.preDestroy() needs to undo everything that happens in WebContainer.postConstruct().

If you want to experiment, the quickest way to try is to do the following:

Copy a web app to domain1/autodeploy/

java -DGlassFish_Interactive=true -jar glassfish.jar

You will see a prompt after sometime. Type: start

It will start GlassFish and deploy the web app. Now you can access the web app. Now type: stop It will stop GlassFish. Then start again. You will see an NPE like this:

[#|2012-07-27T23:35:25.988+0530|WARNING|44.0|javax.enterprise.system.container.web.com.sun.enterprise.web.connector|_ThreadID=10;_ThreadName=main;|Error registering contexts java.lang.NullPointerException at com.sun.enterprise.web.connector.MapperListener.init(MapperListener.java:193) at com.sun.enterprise.web.connector.coyote.PECoyoteConnector.start(PECoyoteConnector.java:549) at org.apache.catalina.startup.Embedded.start(Embedded.java:932) at com.sun.enterprise.web.WebContainer.postConstruct(WebContainer.java:604) at com.sun.hk2.component.AbstractCreatorImpl.inject(AbstractCreatorImpl.java:132) at com.sun.hk2.component.ConstructorCreator.initialize(ConstructorCreator.java:117) at com.sun.hk2.component.AbstractCreatorImpl.get(AbstractCreatorImpl.java:84) at com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:67) at com.sun.hk2.component.EventPublishingInhabitant.get(EventPublishingInhabitant.java:141) at com.sun.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:135) at org.glassfish.internal.data.EngineInfo.getContainer(EngineInfo.java:93) at com.sun.enterprise.v3.server.ApplicationLifecycle.startContainers(ApplicationLifecycle.java:997) at com.sun.enterprise.v3.server.ApplicationLifecycle.setupContainerInfos(ApplicationLifecycle.java:707) at com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLifecycle.java:372) at com.sun.enterprise.v3.server.ApplicationLoaderService.processApplication(ApplicationLoaderService.java:393) at com.sun.enterprise.v3.server.ApplicationLoaderService.postConstruct(ApplicationLoaderService.java:229) at com.sun.hk2.component.AbstractCreatorImpl.inject(AbstractCreatorImpl.java:132) at com.sun.hk2.component.ConstructorCreator.initialize(ConstructorCreator.java:117) at com.sun.hk2.component.AbstractCreatorImpl.get(AbstractCreatorImpl.java:84) at com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:67) at com.sun.hk2.component.EventPublishingInhabitant.get(EventPublishingInhabitant.java:141) at com.sun.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:135) at com.sun.enterprise.v3.server.StartupRunLevelBridge.activate(StartupRunLevelBridge.java:93) at com.sun.enterprise.v3.server.RunLevelBridge.postConstruct(RunLevelBridge.java:110) at com.sun.enterprise.v3.server.StartupRunLevelBridge.postConstruct(StartupRunLevelBridge.java:65) at com.sun.hk2.component.AbstractCreatorImpl.inject(AbstractCreatorImpl.java:132) at com.sun.hk2.component.ConstructorCreator.initialize(ConstructorCreator.java:117) at com.sun.hk2.component.AbstractCreatorImpl.get(AbstractCreatorImpl.java:84) at com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:67) at com.sun.hk2.component.EventPublishingInhabitant.get(EventPublishingInhabitant.java:141) at com.sun.hk2.component.EventPublishingInhabitant.get(EventPublishingInhabitant.java:141) at com.sun.hk2.component.RunLevelInhabitant.get(RunLevelInhabitant.java:110) at com.sun.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:135) at com.sun.enterprise.v3.server.AppServerStartup$StartupInhabitantActivator.activate(AppServerStartup.java:526) at org.jvnet.hk2.component.internal.runlevel.DefaultRunLevelService$Worker.activateRunLevel(DefaultRunLevelService.java:1106) at org.jvnet.hk2.component.internal.runlevel.DefaultRunLevelService$Worker.upActiveRecorder(DefaultRunLevelService.java:1060) at org.jvnet.hk2.component.internal.runlevel.DefaultRunLevelService$Worker.run(DefaultRunLevelService.java:1026) at org.jvnet.hk2.component.internal.runlevel.DefaultRunLevelService$SyncProceedToOp.proceedTo(DefaultRunLevelService.java:1256) at org.jvnet.hk2.component.internal.runlevel.DefaultRunLevelService.proceedTo(DefaultRunLevelService.java:797) at org.jvnet.hk2.component.internal.runlevel.DefaultRunLevelService.proceedTo(DefaultRunLevelService.java:759) at com.sun.enterprise.v3.server.AppServerStartup.proceedTo(AppServerStartup.java:360) at com.sun.enterprise.v3.server.AppServerStartup.run(AppServerStartup.java:254) at com.sun.enterprise.v3.server.AppServerStartup.doStart(AppServerStartup.java:172) at com.sun.enterprise.v3.server.AppServerStartup.start(AppServerStartup.java:163) at com.sun.enterprise.glassfish.bootstrap.GlassFishImpl.start(GlassFishImpl.java:79) at com.sun.enterprise.glassfish.bootstrap.GlassFishDecorator.start(GlassFishDecorator.java:63) at com.sun.enterprise.glassfish.bootstrap.osgi.EmbeddedOSGiGlassFishImpl.start(EmbeddedOSGiGlassFishImpl.java:73) at com.sun.enterprise.glassfish.bootstrap.GlassFishDecorator.start(GlassFishDecorator.java:63) at com.sun.enterprise.glassfish.bootstrap.osgi.OSGiGlassFishImpl.start(OSGiGlassFishImpl.java:71) at com.sun.enterprise.glassfish.bootstrap.GlassFishMain$Launcher.startConsole(GlassFishMain.java:129) at com.sun.enterprise.glassfish.bootstrap.GlassFishMain$Launcher.launch(GlassFishMain.java:115) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at com.sun.enterprise.glassfish.bootstrap.GlassFishMain.main(GlassFishMain.java:97) at com.sun.enterprise.glassfish.bootstrap.ASMain.main(ASMain.java:55)

| #] |

Please treat this urgently.

Affected Versions

[4.0]

[glassfishrobot]

• Issue Imported From: https://github.com/javaee/glassfish/issues/18955
• Original Issue Raised By:@glassfishrobot
• Original Issue Assigned To: @amyroh
• Original Issue Closed By:@glassfishrobot

[glassfishrobot]

@glassfishrobot Commented @shingwaichan said: I have the latest workspace. I do not see the NPE mentioned above. Can you try with the latest workspace?

[glassfishrobot]

@glassfishrobot Commented ss141213 said: Good to know the NPE does not appear with latest code. Does the webapp actually work after embedded server is restarted?

[glassfishrobot]

@glassfishrobot Commented @shingwaichan said: The connector is not working after running "stop" and "start" again. Can you clarify what "-DGlassFish_Interactive=true", "stop" and "start" are doing? Is it a new feature? I do not see WebContainer.preDestroy is called from the debugger.

[glassfishrobot]

@glassfishrobot Commented ss141213 said: Shingwai,

That's what was my observation as well. Connectors are not working after embedded server is restarted. I do believe it's a problem with WebContainer.preDestroy() which is not undoing everything that has happened in postConstruct(). I applied the following patch in my local workspace to fix the connectors issue:

ss141213@Sahoo:/space/ss141213/WS/gf/trunk$ git diff appserver/web diff --git a/appserver/web/web-glue/src/main/java/com/sun/enterprise/web/WebContainer.java b/appserver/web/web-glue/src/main/java/com/sun/enterprise/web/WebContainer.java index 68f6c37..eed42ac 100644 — a/appserver/web/web-glue/src/main/java/com/sun/enterprise/web/WebContainer.java +++ b/appserver/web/web-glue/src/main/java/com/sun/enterprise/web/WebContainer.java @@ -634,7 +634,12 @@ public class WebContainer implements org.glassfish.api.container.Container, Post }

public void preDestroy() {

• // FIX ME: preDestroy should be just the ooposite of postConstruct, but I am not sure why it is not try {
• for (WebConnector c : new ArrayList(connectorMap.values()))

{ + deleteConnector(c); + System.out.println("WebContainer.preDestroy " + c); + }

_embedded.stop(); } catch (LifecycleException le) { _logger.log(Level.SEVERE,

But, even then something is broken. There after I decided to open this bug.

To answer your question about -DGlassFish_Interactive, start, stop, no, they have been there in the code base for a couple of years now. We internally use them to try out embedded features. It's so much simpler to use them rather than having to write a new program to describe a bug.

[glassfishrobot]

@glassfishrobot Commented @shingwaichan said: Is it related to embedded restart. Assign to Amy for further investigation.

[glassfishrobot]

@glassfishrobot Commented @amyroh said: Sahoo,

Which embedded jar are you using? I can't start the embedded server using the latest trunk.

org.glassfish.embeddable.GlassFishException: com.sun.enterprise.module.bootstrap.BootException: Cannot find main module : no such module at com.sun.enterprise.glassfish.bootstrap.StaticGlassFishRuntime.newGlassFish(StaticGlassFishRuntime.java:155) .... Caused by: com.sun.enterprise.module.bootstrap.BootException: Cannot find main module : no such module at com.sun.enterprise.module.bootstrap.Main.findStartupService(Main.java:234) at com.sun.enterprise.glassfish.bootstrap.StaticGlassFishRuntime.newGlassFish(StaticGlassFishRuntime.java:123)

[glassfishrobot]

@glassfishrobot Commented ss141213 said: Amy,

As I mentioned in the bug notes, I am using the regular glassfish.jar that exists in glassfish/modules dir. Just follow the instruction provided in the bug description. Shingwai has already reproduced it.

Thanks, Sahoo

[glassfishrobot]

@glassfishrobot Commented @amyroh said: I confirm that WebContainer#preDestroy/postConstruct are never invoked during GlassFish#stop/start. Shouldn't these be called from GlassFish? AppServerContext#serverStopping() is never invoked during server stop and the old WebContainer is used rather than a new WebContainer getting created via WebContainer#postConstruct.

[glassfishrobot]

@glassfishrobot Commented ss141213 said: Amy,

The behavior you have observed must be to do with recent HK2 2.1.x integration. But, when I filed the bug, I used a version of glassfish that used HK2 2.0.x and there I did notice preDestroy() getting called. Since you should not be worrying about these details, I recommend you using GlassFish 3.1.2 to reproduce the bug and fix it. Or, you can use 4weeks older trunk builds of GlassFish as well. Upto you. The fix will be equally applicable to most recent workspace.

Thanks much, Sahoo

[glassfishrobot]

@glassfishrobot Commented @amyroh said: I've put in an incremental fix (55377) but this needs to be investigated further.

The trunk needs to be fixed so that WebContainer#preDestroy/postConstruct are invoked during GlassFish#stop/start.

[glassfishrobot]

@glassfishrobot Commented @amyroh said: If I manually invoke preDestroy/postConstruct during glassfish restart using EventListener, I can access 8080 after restart with the patch but I still get 404 for previously deployed webapps so something else is broken here. I'll investigate this further when I get back.

Assigning to hk2 so WebContainer is destroyed during shutdown so it can be started via postConstruct during restart. Embedded server start is also broken as described in #18989. Please assign it back to me once AppServerContext is fixed to invoke serverStopping() -> WebContainer preDestroy.

[glassfishrobot]

@glassfishrobot Commented @jwells131313 said: I have recently added the ability for @Singleton scoped objects to be shutdown on the shutdown of the ServiceLocator. I believe that this must therefore be happening because the shutdown of the embedded server isn't happening properly, which is is Mason's area of expertise.

[glassfishrobot]

@glassfishrobot Commented @mtau11 said: After Revision 55766, WebContainer.preDestroy is being called on shutdown

[glassfishrobot]

@glassfishrobot Commented @amyroh said: I confirm that WebContainer.preDestroy gets invoked on shutdown as expected.

However, when restarting GrizzlyService.postConstruct is never called and the mappers are not starting.

GrizzlyService.postConstruct is called as expected when starting glassfish for the first time. Stopping is calling GrizzlyService.preDestroy as expected but restarting glassfish never invokes GrizzlyService.postConstruct.

[glassfishrobot]

@glassfishrobot Commented @mtau11 said: After revision 55791, GrizzlyService.preDestroy/postConstruct will be called.

[glassfishrobot]

@glassfishrobot Commented @amyroh said: After Ryan's fix (svn 56001) in GlassfishNetworkListener.stop, restart is working as expected.

[glassfishrobot]

@glassfishrobot Commented This issue was imported from java.net JIRA GLASSFISH-18955

[glassfishrobot]

@glassfishrobot Commented Reported by ss141213

[glassfishrobot]

@glassfishrobot Commented Marked as fixed on Monday, September 17th 2012, 12:52:38 pm