eclipse-ee4j / glassfish

Eclipse GlassFish
https://eclipse-ee4j.github.io/glassfish/
378 stars 144 forks source link

Durable subscription created when nondurable one requested. #6580

Closed glassfishrobot closed 14 years ago

glassfishrobot commented 15 years ago

I am trying to configure the clustered cache of EclipseLink to use JMS in Glassfish v2.1-b54. I created a Glassfish cluster with 2 nodes and then:

When I run my application standalone (with the various Glassfish jars), it connects to the cluster and creates nondurable subscriptions so it all works fine. However, when I run the application inside Glassfish, for some reason durable subscriptions are created. This causes the application to fail because the client ID has not been set as shown in the stacktrace below[1]. The EclipseLink code that creates the subscription is (and the stacktrace shows that too):

this.subscriber = topicSession.createSubscriber(topic);

The javadoc for that method says:

"Creates a nondurable subscriber to the specified topic."

So, why is a durable subscriber being created? I don't think it's an issue with the administered object because it works fine when I run the code from a standalone application (which looks up the same administered object).

Another information that might be useful is that I am using com.sun.appserv.naming.S1ASCtxFactory as the initial context factory name.

broker error occurred. :[412] [B4135]: Cannot add durable consumer null. No ClientID was set on connection. user=guest, broker=localhost:37676(59279) at org.eclipse.persistence.exceptions.EntityManagerSetupException.failedWhileProcessingProperty(EntityManagerSetupException.java:178) at org.eclipse.persistence.internal.jpa.EntityManagerSetupImpl.processSessionCustomizer(EntityManagerSetupImpl.java:1222) at org.eclipse.persistence.internal.jpa.EntityManagerSetupImpl.updateServerSession(EntityManagerSetupImpl.java:1198) at org.eclipse.persistence.internal.jpa.EntityManagerSetupImpl.deploy(EntityManagerSetupImpl.java:237) at org.eclipse.persistence.internal.jpa.EntityManagerFactoryImpl.getServerSession(EntityManagerFactoryImpl.java:69) at org.eclipse.persistence.internal.jpa.EntityManagerFactoryImpl.createEntityManagerImpl(EntityManagerFactoryImpl.java:118) at org.eclipse.persistence.internal.jpa.EntityManagerFactoryImpl.createEntityManagerImpl(EntityManagerFactoryImpl.java:112) at org.eclipse.persistence.internal.jpa.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:100) at org.springframework.orm.jpa.JpaTransactionManager.createEntityManagerForTransaction(JpaTransactionManager.java:392) at org.springframework.orm.jpa.JpaTransactionManager.doBegin(JpaTransactionManager.java:320) ... 42 more Caused by: Exception [EclipseLink-22112] (Eclipse Persistence Services - 1.0.1 (Build 20080905)): org.eclipse.persistence.exceptions.RemoteCommandManagerException Exception Description: Could not create local JMS connection with Topic jms/topic/eclipseLink, Topic Factory jms/connFactory/eclipseLink/all, and Context properties

{org.omg.CORBA.ORBInitialPort=33700, java.naming.provider.url=iiop://localhost:33700, java.naming.factory.initial=com.sun.appserv.naming.S1ASCtxFactory, java.naming.security.principal=guest, dedicated.connection=true, java.naming.security.credentials=24D77DC031B68CE91A372A5A33219416}

Internal Exception: com.sun.messaging.jms.JMSException: [ADD_CONSUMER_REPLY(15)] [C4036]: A broker error occurred. :[412] [B4135]: Cannot add durable consumer null. No ClientID was set on connection. user=guest, broker=localhost:37676(59279) at org.eclipse.persistence.exceptions.RemoteCommandManagerException.errorCreatingLocalJMSConnection(RemoteCommandManagerException.java:177) at org.eclipse.persistence.sessions.coordination.jms.JMSTopicTransportManager.createConnection(JMSTopicTransportManager.java:105) at org.eclipse.persistence.sessions.coordination.jms.JMSTopicTransportManager.createLocalConnection(JMSTopicTransportManager.java:78) at org.eclipse.persistence.sessions.coordination.jms.JMSTopicTransportManager.createConnections(JMSTopicTransportManager.java:147) at org.eclipse.persistence.sessions.coordination.RemoteCommandManager.initialize(RemoteCommandManager.java:149) at com.likecube.jpa.eclipselink.EclipseLinkSessionCustomizer.customize(EclipseLinkSessionCustomizer.java:94) at org.eclipse.persistence.internal.jpa.EntityManagerSetupImpl.processSessionCustomizer(EntityManagerSetupImpl.java:1220) ... 50 more Caused by: com.sun.messaging.jms.JMSException: [ADD_CONSUMER_REPLY(15)] [C4036]: A broker error occurred. :[412] [B4135]: Cannot add durable consumer null. No ClientID was set on connection. user=guest, broker=localhost:37676(59279) at com.sun.messaging.jmq.jmsclient.ProtocolHandler.throwServerErrorException(ProtocolHandler.java:3982) at com.sun.messaging.jmq.jmsclient.ProtocolHandler.addInterest(ProtocolHandler.java:2307) at com.sun.messaging.jmq.jmsclient.WriteChannel.addInterest(WriteChannel.java:88) at com.sun.messaging.jmq.jmsclient.ConnectionImpl.addInterest(ConnectionImpl.java:1338) at com.sun.messaging.jmq.jmsclient.Consumer.registerInterest(Consumer.java:145) at com.sun.messaging.jmq.jmsclient.MessageConsumerImpl.addInterest(MessageConsumerImpl.java:170) at com.sun.messaging.jmq.jmsclient.MessageConsumerImpl.init(MessageConsumerImpl.java:157) at com.sun.messaging.jmq.jmsclient.MessageConsumerImpl.(MessageConsumerImpl.java:115) at com.sun.messaging.jmq.jmsclient.TopicSubscriberImpl.(TopicSubscriberImpl.java:104) at com.sun.messaging.jmq.jmsclient.UnifiedSessionImpl.createSubscriber(UnifiedSessionImpl.java:313) at com.sun.messaging.jmq.jmsclient.UnifiedSessionImpl.createSubscriber(UnifiedSessionImpl.java:274) at com.sun.messaging.jms.ra.SessionAdapter.createSubscriber(SessionAdapter.java:312) at org.eclipse.persistence.internal.sessions.coordination.jms.JMSTopicRemoteConnection.(JMSTopicRemoteConnection.java:78) at org.eclipse.persistence.sessions.coordination.jms.JMSTopicTransportManager.createConnection(JMSTopicTransportManager.java:101) ... 55 more

Environment

Operating System: All Platform: All

Affected Versions

[v2.1]

glassfishrobot commented 6 years ago
glassfishrobot commented 15 years ago

@glassfishrobot Commented ijuma said: I did some debugging and found out a few more things. When it works fine (outside the container), the properties of ReadWritePacket are:

{JMQDestType=2, JMQDestination=jms.topic.eclipseLink, JMQAckMode=1, JMQSessionID=3998665504591133696, JMQReconnect=false, JMQShare=false, requestMetaData=ConnectionID=3998665504588894720, SessionID=3998665504591133696, ConsumerID=null, DestName=jms.topic.eclipseLink, JMQNoLocal=false, JMQSize=1000}

When it doesn't work (inside the container), the properties are:

{JMQDestType=2, JMQDestination=jms.topic.eclipseLink, JMQAckMode=1, JMQSessionID=3998665504571367936, JMQReconnect=false, JMQShare=true, requestMetaData=ConnectionID=3998665504569793024, SessionID=3998665504571367936, ConsumerID=null, DestName=jms.topic.eclipseLink, JMQNoLocal=false, JMQSize=1000}

Note that the only relevant difference is JMQShare=true instead of JMQShare=false. Is it possible that the broker would take a JMQShare=true to imply a durable subscription?

The reason why JMQShare is true inside the container is that the method ConnectionImpl.setRANamespaceUID is called from ConnectionAdapter:

if (ra != null) { inACC = ra.getInAppClientContainer(); if (ra.getInClusteredContainer())

{ xac.setRANamespaceUID(ra._getRAUID()); }

}

setRANamespaceUID among other things sets imqEnableSharedSubscriptions to true.

glassfishrobot commented 15 years ago

@glassfishrobot Commented ijuma said: I looked into the broker code and the problem occurs in ComsumerHandler.createConsumer:

else if ((wildcard || !d.isQueue()) && shared) { if (clientid == null)

{ throw new BrokerException( Globals.getBrokerResources().getKString( BrokerResources.X_NO_CLIENTID, durablename), BrokerResources.X_NO_CLIENTID, null, Status.PRECONDITION_FAILED); }

The "shared" variable represents the JMQShare property so this condition becomes true because of that.

glassfishrobot commented 15 years ago

@glassfishrobot Commented gregn123 said: We also encountered this problem (attempt to create non-durable subscriber fails with "Cannot add durable consumer null" error).

We found that calls to TopicSession.createSubscriber() fail in the following circumstance:

(1) The calling application is deployed to a Glassfish cluster AND (2) the cluster's JMS Service type is REMOTE or LOCAL (NOTE: if it is EMBEDDED, the application works fine!)

I have attached a simple test application, TestJMS.ear, which reproduces the problem on a freshly installed Glassfish V2.1 (NOTE: setup Glassfish using "ant -f setup-cluster.xml") The test application consists of an EJB and client. The EJB has a "runTest" method, which creates a TopicConnection and TopicSession, then attempts to create a non-durable TopicSubscriber by invoking createSubscriber() on the TopicSession. If this succeeds, "PASSED" is returned, otherwise "FAILED" is returned. Source code for EJB and client is included in TestJMS.ear.

Follow the procedure below to deploy and run the test application on Windows:

(1) Unzip TestJMS.zip to (say) your C: drive.

(2) cd c:\TestJMS

(3) Run the following supplied batch file:

gf_create_and_start_cluster.bat

NOTE1: modify the GF_HOME environment variable setting in the batch file to point to your Glassfish V2.x installation NOTE2: UNIX users can easily convert the batch file to a shellscript

This batch file creates a cluster, sets the cluster's JMS Service type to REMOTE, starts an MQ broker, starts the cluster and finally creates necessary JMS resources for the test application.

(4) Deploy the test application:

asadmin deploy --target cluster1 TestJMS\lib\TestJMS.ear

(5) Run the client application (that invokes the EJB):

appclient -client TestJMS\lib\TestJMS.ear -xml sun-acc.xml

The output will be: Test result: FAILED

In the MQ.BROKER window, a message similar to the following will be displayed:

[19/Nov/2008:16:49:36 EST] WARNING [B2106]: Creation of consumer from connection guest@127.0.0.1:59045 on the auto-created destination TestTopic failed:com.sun.messaging.jmq.jmsserver.util.BrokerException: [B4135]: Cannot add durable consumer null. No ClientID was set on connection.

and in the cluster's server.log file there will be a message similar to the following:

|2008-11-19T16:49:36.822+1100|WARNING|sun-appserver2.1|javax.enterprise.system.stream.err|_ThreadID=20;_ThreadName=p:

thread-pool-1; w: 5;_RequestID=419e90c3-01d2-44bf-81c8-ae4df1ffa78e;|Exception occurred: com.sun.messaging.jms.JMSException: [ADD_CONSUMER_REPLY(15)] [C4036]: A broker error occurred. :[412] [B4135]: Cannot add durable consumer null. No ClientID was set on connection. user=guest, broker=localhost:37676(58839)|#]

If you re-run the application with the cluster's JMS Service type set to EMBEDDED instead of REMOTE, the application works fine:

asadmin stop-cluster cluster1 asadmin set cluster1.jms-service.type=EMBEDDED asadmin start-cluster cluster1 appclient -client TestJMS\lib\TestJMS.ear -xml sun-acc.xml

In this case, the test output is: Test result: PASSED

and the cluster's server.log file contains the following application output:

[#|2008-11-19T17:52:26.329+1100|INFO|sun-appserver2.1|javax.enterprise.system.stream.out|_ThreadID=17;_ThreadName=p: thread-pool-1; w: 5;|Create TopicSubscriber succeeded!!!!|#]

glassfishrobot commented 15 years ago

@glassfishrobot Commented gregn123 said: Created an attachment (id=2109) Test application which reproduces the createSubscriber() problem

glassfishrobot commented 15 years ago

@glassfishrobot Commented sanandal said: "Reclassifying as P4 because this issue is not deemed "must fix" for this v2.1 release whose primary release driver is SailFin. This issue will be scrubbed after this release and will be given the right priority for the next release."

glassfishrobot commented 15 years ago

@glassfishrobot Commented lindaschneider said: To clarify - the behavior is not that we are converting your consumer to be durable (the message is misleading) its that we are "sharing" them across the cluster.

Also, this behavior is only turned on for a glassfish cluster.

What does share mean ???

For durables what it means is: if you create two durables with the same name and same clientID on different servers ... they will be considered a "single" durable. e.g.: You send 10 messages to topic foo [and create two subscribers with a clientID of cid1 and a durable name of dname] [gf instance 1: cid1, dname on foo] gets 5 messages [gf instance 2: cid1, dname on foo] - gets 5 messages

For non-durable subscribers: If you create two subscribers on the same topic with the same clientID, they will share messages (same as the above durables case)

If no clientID, the durables can't be shared and the exception is thrown.

The correct solution is to provide hooks to allow this behavior to be turned off by the glassfish user

glassfishrobot commented 15 years ago

@glassfishrobot Commented cmathrusse said: To clarify, we are not creating Durable subscriptions. EclipseLink is simply subscribing to the topic (Non-Durable). But because GlassFish is clustered and the JMS servers are Remote and Clustered, we are receiving this exception message.

No ClientID can be provided because per the JMS API spec, when invoking createSubscriber(...) on the TopicSession, there is no way to provide a ClientID. Only createDurableSubscriber(...) provides a means to set the ClientID.

glassfishrobot commented 15 years ago

@glassfishrobot Commented lindaschneider said: If you are creating your own connection there is a way in the specification to create a client ID.

Please see:

http://java.sun.com/javaee/6/docs/api/javax/jms/Connection.html#setClientID(java.lang.String

The rules are that you must set this attribute before doing anything with the connection (which is covered in the javadoc). There are no negative ramifications on setting this property on a connection which has non-durable subscribers when you are running in a standalone environment.

If you are running using a connection factory object created BY the application server, you can set the connection factory through multiple methods:

see http://docs.sun.com/app/docs/doc/820-4335/abljw?l=en&a=view&q=ClientId

Note: the proposed changes only allow you to skip the setting the clientID step. There is nothing that prevents you from setting it (outside of code which is outside of your control doesn't set it)

glassfishrobot commented 15 years ago

@glassfishrobot Commented cmathrusse said: OK, I stand corrected about setting the ClientID on the Connect itself. (good to know) but that is out of our control, unless I make code changed to EclipseLink. (I would prefer to not have to do that)

Pertaining to supplying the ClientID on the ConnectionFactory itself, I'm a bit unclear as to where I can set this as a property. Is this referring to adding a property to the ConnectionFactory itself, by simply adding a new property, naming it ClientId, and specifying a value?

If this is true, what happens in a clustered environment? Won't all the instances be using the same ClientID?

Thanks for the help...

glassfishrobot commented 15 years ago

@glassfishrobot Commented cmathrusse said: Setting the ClientID on the ConnectionFactory does not resolve our issue, not unless there is only going to be one connection acquired from the ConnectionFactory. As soon as another connection is requested an exception is raised due to the ClientID is already in use.

EclipseLink uses 2 connections: One to publish and one to subscribe. If I set the ClientID on the ConnectionFactory, the first connection acquired is returned without error, but when the second connection is requested and exception is raised, as described about. So this is not a usable work-around for our issue.

I've also attempted to set the ClientID on the connection following the call to connectionFactory.createTopicConnection();, but this also raised an exception:

com.sun.messaging.jms.JMSException: MQRA:CA:Unsupported-setClientID()

I think this is because setting the ClientID is only allowed in the Application Client Container. So there is no viable way to set the ClientID on the connection inside the application and no way to assign a unique ClientID to each connection requested from the ConnectionFactory. So I am out of work-arounds to this issue.

glassfishrobot commented 15 years ago

@glassfishrobot Commented nzjess said: Just thought I'd mention the work-around I'm using for this... if you want a non-durable connection, but want multiple clients (eg. in a cluster), then this is how you can achieve this (example code is given in terms of a Spring MessageListenerContainer, but you can adapt as required for your application):

By the way, this is about as dirty as it gets, and may even have some unforseen side-effects! But, it does work (around), at least for Glassfish V2.1.1-b17. Disclaimer: I haven't actually used this in production.

import java.lang.reflect.Field;

import javax.jms.Connection; import javax.jms.JMSException;

import org.springframework.jms.listener.DefaultMessageListenerContainer;

public class MessageListenerContainer extends DefaultMessageListenerContainer {

private static Field ConnectionAdapter_xac; private static Field ConnectionImpl_imqEnableSharedSubscriptions;

static { // work-around for: https://glassfish.dev.java.net/issues/show_bug.cgi?id=6580 try

{ Class ConnectionAdapter = Class.forName("com.sun.messaging.jms.ra.ConnectionAdapter"); ConnectionAdapter_xac = ConnectionAdapter.getDeclaredField("xac"); ConnectionAdapter_xac.setAccessible(true); Class ConnectionImpl = Class.forName("com.sun.messaging.jmq.jmsclient.ConnectionImpl"); ConnectionImpl_imqEnableSharedSubscriptions = ConnectionImpl.getDeclaredField("imqEnableSharedSubscriptions"); ConnectionImpl_imqEnableSharedSubscriptions.setAccessible(true); }

catch (Exception e)

{ e.printStackTrace(System.out); }

}

@Override protected Connection createConnection() throws JMSException { // work-around for GlassFish bug Connection connection = super.createConnection(); if (EnvironmentHints.isGlassFish()) { try { if (connection != null)

{ Object xac = ConnectionAdapter_xac.get(connection); ConnectionImpl_imqEnableSharedSubscriptions.set(xac, false); }

else

{ System.out.println("connection is null"); }

} catch (Exception e)

{ e.printStackTrace(System.out); }

} return connection; }

@Override public void setClientId(String clientId)

{ // work-around for 'In Application Container' restriction super.setClientId(clientId); }

}

glassfishrobot commented 15 years ago

@glassfishrobot Commented nzjess said: Ooops, in the above code, the setClientId() method just needs to not call super, if you want to avoid the 'In Application Container' restriction - you get the idea.

glassfishrobot commented 15 years ago

@glassfishrobot Commented cmathrusse said: Thanks for the code example. This would work well if I were to use Spring for as the MmessageListener, but unfortunately I would need to make code changes to EclipseLink to support the use of this as well as setting the ClientID, something I would like to avoid. For now, I guess the only solution that I have is to by-pass the container all together when it comes to the ConnectionFactory to the JMS Server. I'll need to go directly to the Remote JMS server.

This too is very dirty. EclipseLink wants to acquire a ConnectionFactory and Topic from JNDI by performing a lookup from the InitialContext. So now I am forced into implementing a class that mimics the functionality that I EclipseLink needs to perform a lookup of these two objects, which will be stored in the Spring Container, to simply work around an issue in GlassFish that should have been addressed a while back.

glassfishrobot commented 15 years ago

@glassfishrobot Commented lindaschneider said: Please forgive the fact that I know absolutely nothing about how EclipseLink works but ...

If EclipseLink is retrieving the JMS connection factory object from LDAP (and not creating in manually) ...

Can't you just edit the specific connection factory to set a clientID ???

you would use imqobjmgr to point to the right object

If that doesn't work ... you might even be able to override it on the command line when you start the app server e.g. pass in -DimqConfiguredClientID=MyID

[Note: I can't be sure because I've never tried that specific property]


Also ... I may be dense (I am very JMS spec focused) but I'm really not understanding why any of this is an issue that can not be worked around???

Yes - in this clustering case clientID needs to be set for Topics (and long term this needs to be able to be turned off - although this is different/fixed in v3). This isn't something that will be backported to v2.X.

However with the current situation:

What is the case you are running into where none of these work, and why not ??

Can you specify how in your mind it "should work" ??

glassfishrobot commented 15 years ago

@glassfishrobot Commented cmathrusse said: I'm sorry if I am missing something here pertaining to the configuration of the ConnectionFactory (Managed Object) in the GlassFish container but none of these options seems to work as needed. Let me try and address them.

EclipseLink is designed to allow us to utilize JMS for Cache Coordination. It does this in a standard way by allowing us to specify the properties that are used to acquire an InitiaContext. We also specify the JNDI path to the TopicConnectionFactory and the Topic itself.

Example:

public void customize(Session session) throws Exception { RemoteCommandManager rcm = new RemoteCommandManager((CommandProcessor) session); java.util.Properties props = new java.util.Properties(); props.put(Context.PROVIDER_URL, getProviderURL()); props.put(Context.INITIAL_CONTEXT_FACTORY, getInitialContextFactory());

JMSTopicTransportManager tm = new JMSTopicTransportManager(rcm); tm.setLocalContextProperties(props); tm.setRemoteContextProperties(props); tm.setTopicName(getJmsTopicName()); tm.setTopicConnectionFactoryName(getJmsConnectionFactoryName()); tm.setUserName(getUserName()); tm.setPassword(getPassword()); ....

EclipseLink allows us to specify the local and remote contexts. Local is for creating a subscriber and remote is for creating a publisher. (Usually both Local and Remote would be the same.) This is due to EclipseLink publishing entity changed to the topic. The subscriber receives those changes to apply to entities within the cache or to invalidate those entities. Both Local and Remote are treated separately, so each will have a reference to a separate connection.

Now, if I were to set the ClientID on the ConnectionFactory that is managed in JNDI by GlassFish, the ClientID is set on all connections created. The problem with this is that it only works correctly for the first connection. The second connection created will throw an exception because the ClientID is already in use. (I know for a fact because I tried it)

So it makes sense that the ClientID must be generated and unique. Seems to me I could set it easily after creating the connection. Something like:

TopicConnection topicConnection=connectionFactory.createTopicConnection(); topicConnection.setClientID(getClientID()); ...

But this too fails due to a little piece of code in GlassFish that prevents the ClientID from being set outside of the Application Container:

Referring to com.sun.messaging.jms.ra.ConnectionAdapter :

public void setClientID(String clientId) throws JMSException { _loggerJC.entering(_className, "setClientID()", clientId); //System.out.println("MQRA:CA:setClientID-to-"+clientId); if (!inACC)

{ throw new com.sun.messaging.jms.JMSException( "MQRA:CA:Unsupported-setClientID()"); }

checkClosed(); xac.setClientID(clientId); }

The above code prevents me from setting the ClientID from within my application code. (Again, I've tried..) When I call setClientID(...) I get the JMSException listed above.

So if you know of a way that I can set the ClientID on the Connection or ConnectionFactory that will by-pass all of the Exceptions that I am receiving then please share it with me. At this point I'm having to by-pass the managed objects completely due to the Exceptions that are preventing me from using them.

Keep in mind, none of this would be an issue if GlassFish wasn't raising the exception when sharing the Non-Durable Connection in the clustered environment.

Thanks for the help...

glassfishrobot commented 15 years ago

@glassfishrobot Commented kumara said: Changing version from 9.1.1 to v2.1 to reflect new name/version.

glassfishrobot commented 14 years ago

@glassfishrobot Commented nigeldeakin said: The problems reported in this issue occur when trying to use a non-durable topic subscription in a Glassfish cluster. The MQ resource adapter (JMSRA) expects ClientID to be set so that it can share the same subscription amongst all instances of Glassfish, and by the laws of JavaEE you can't explicitly set ClientID outside the application client container. Users are therefore running into "ClientID not set" errors, which are getting misreported as "Cannot add durable consumer, clientID not set".

In MQ 4.4 (e.g. in Glassfish 2.1.1) you can disable the sharing of non-durable topic subscriptions by setting the property useSharedSubscriptionInClusteredContainer=false on the ManagedConnectionFactory (which is what you're really dealing with when you configure a MQ connection factory in Glassfish). You can set this in the Glassfish admin console.

If you set this property to false, then you should no longer get "ClientID not set" errors (which are being misreported as "Cannot create durable subscription".

This property is documented in the MQ 4.4 admin guide here: http://docs.sun.com/app/docs/doc/821-0027/aeoon?l=en&a=view

glassfishrobot commented 15 years ago

@glassfishrobot Commented File: TestJMS.zip Attached By: gregn123

glassfishrobot commented 15 years ago

@glassfishrobot Commented Was assigned to sats

glassfishrobot commented 7 years ago

@glassfishrobot Commented This issue was imported from java.net JIRA GLASSFISH-6580

glassfishrobot commented 15 years ago

@glassfishrobot Commented Reported by ijuma

glassfishrobot commented 14 years ago

@glassfishrobot Commented Marked as fixed on Monday, May 31st 2010, 8:18:16 pm