Generic Connector Communication Error

[spaskovich]

Hello -

In certain situations (when I have to pull a decent amount of data from my mysql database) I receive the following error:

Generic Connector Communication Error: Please check and adjust your setup: Ensure that Tomcat is running on given host and port. If this is a timeout error consider adjusting IIS timeout by changing executionTimeout attribute in web.config (see manual).

Note: When my users only have around 50 or so rows of data being pulled, everything works fine.

My catalina log shows the following:

15-Mar-2018 11:57:44.686 SEVERE [ajp-nio-8009-exec-4] org.apache.coyote.ajp.AjpMessage.checkOverflow Overflow error for buffer adding [88728] bytes at position [552] java.lang.ArrayIndexOutOfBoundsException at org.apache.coyote.ajp.AjpMessage.checkOverflow(AjpMessage.java:258) at org.apache.coyote.ajp.AjpMessage.appendBytes(AjpMessage.java:225) at org.apache.coyote.ajp.AjpMessage.appendByteChunk(AjpMessage.java:209) at org.apache.coyote.ajp.AjpMessage.appendBytes(AjpMessage.java:191) at org.apache.coyote.ajp.AjpProcessor.prepareResponse(AjpProcessor.java:1099) at org.apache.coyote.ajp.AjpProcessor$SocketOutputBuffer.doWrite(AjpProcessor.java:1482) at org.apache.coyote.Response.doWrite(Response.java:541) at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:351) at org.apache.catalina.connector.OutputBuffer.appendByteArray(OutputBuffer.java:785) at org.apache.catalina.connector.OutputBuffer.append(OutputBuffer.java:714) at org.apache.catalina.connector.OutputBuffer.writeBytes(OutputBuffer.java:391) at org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java:369) at org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:96) at org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:89) at lucee.runtime.writer.CFMLWriterImpl.close(CFMLWriterImpl.java:403) at lucee.commons.io.IOUtil.closeEL(IOUtil.java:467) at lucee.runtime.PageContextImpl.close(PageContextImpl.java:723) at lucee.runtime.PageContextImpl.release(PageContextImpl.java:582) at lucee.runtime.CFMLFactoryImpl.releaseLuceePageContext(CFMLFactoryImpl.java:209) at lucee.runtime.engine.CFMLEngineImpl._service(CFMLEngineImpl.java:1108) at lucee.runtime.engine.CFMLEngineImpl.serviceCFML(CFMLEngineImpl.java:1039) at lucee.loader.engine.CFMLEngineWrapper.serviceCFML(CFMLEngineWrapper.java:102) at lucee.loader.servlet.CFMLServlet.service(CFMLServlet.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:742) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:504) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342) at org.apache.coyote.ajp.AjpProcessor.service(AjpProcessor.java:486) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:790) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1459) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748)

OS: Windows Server 2016 (fully updated) Lucee: 5.2.5.20 MySQL: latest version

This is happening off a fresh server setup, default lucee install, default mysql install.

I've gone through the documentation and tried various settings...right now my server.xml for the AJP connector is:

BonCodeAJP13.settings is:

8009 localhost 0 0 0 False 65536

Note: This application was working flawlessly in ACF 8.01, I recently upgraded for security reasons (and to use TLS 1.2 among other things). So I'm rather certain the code is solid.

Please let me know if you need any additional information...I'm completely stuck at this point. Appreciate any assistance in advance!

-Steve

[Bilal-S]

Steve, can you send me the server.xml from your tomcat install. Especially the line that defines AJP protocol="AJP/1.3"

[spaskovich]

Sorry, the code got blanked out when I originally added it to the post...here you go (both server.xml and boncode.settings

`right now my server.xml for the AJP connector is: Connector port="8009" protocol="AJP/1.3" redirectPort="8443" maxThreads="1000" keepAliveTimeout="-1" connectionTimeout="121000" packetSize="65536"

BonCodeAJP13.settings is:

Settings> Port>8009 Server>localhost
MaxConnections>0 LogLevel>0 FlushThreshold>0 EnableRemoteAdmin>False PacketSize>65536 Settings

[omrisk]

I'm also having this exact issue, my settings for AJP connector in the tomcat server.xml is: <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" URIEncoding="UTF-8" tomcatAuthentication="false"/>

and the bonCode.settings are `

<Server></Server>
<MaxConnections></MaxConnections>
<FlushThresholdTicks>0</FlushThresholdTicks>
<FlushThresholdBytes></FlushThresholdBytes>
<LogLevel></LogLevel>
<LogDir></LogDir>
<LogFile></LogFile>
<EnableRemoteAdmin></EnableRemoteAdmin>
<AutoCompression></AutoCompression>
<EnableHeaderDataSupport></EnableHeaderDataSupport>
<HeaderWhitelist></HeaderWhitelist>
<HeaderBlacklist></HeaderBlacklist>
<ReadTimeOut></ReadTimeOut>
<WriteTimeOut></WriteTimeOut>
<ForceSecureSession></ForceSecureSession>
<ResolveRemoteAddrFrom></ResolveRemoteAddrFrom>
<AllowEmptyHeaders></AllowEmptyHeaders>
<PathInfoHeader></PathInfoHeader>
<TomcatConnectErrorURL></TomcatConnectErrorURL> 
<EnableHTTPStatusCodes></EnableHTTPStatusCodes>
<TCPClientErrorMessage></TCPClientErrorMessage>
<TCPStreamErrorMessage></TCPStreamErrorMessage>
<PathPrefix></PathPrefix>
<PacketSize></PacketSize>
<EnableAdobeMode></EnableAdobeMode>
<EnableClientFingerPrint></EnableClientFingerPrint> 
<FPHeaders></FPHeaders>
<SkipIISCustomErrors></SkipIISCustomErrors> 
<LogIPFilter></LogIPFilter>
<EnableAggressiveGC></EnableAggressiveGC>
<RequestSecret></RequestSecret>
<ModCFMLSecret></ModCFMLSecret>

`

Any help would be greatly appreciated.

[Bilal-S]

The error is thrown by tomcat and I have not reproduced it so far. Can set the number of bytes to a lower level: 65530 instead of 65536. You will need to change both server.xml and BonCodeAJP13.settings files to this new value. @omrisk Why do you have all the empty settings?

[Bilal-S]

@Steve, is there a minimum case you have that I can use to reproduce?

[omrisk]

@Bilal-S , I've removed the empty settings (we have some machines running older versions of the connector and they needed the settings defined for some reason). I'll try your suggested configurations for a few days and post an update.

[spaskovich]

I tried literally every single permutation of settings (literally took a couple days) and couldn’t get it working. Since this is a production server I had to roll the project back (back to ACF8.01 & Microsoft Access). I was losing money and customer too rapidly I keep messing around, I’ve noticed there are many people who encounter this problem but many don’t have a solution. It’s unfortunate as I was excited to move over to Lucee as I refuse to pay Adobe’s ridiculous price for ACF2016 (I’m a small business with tight margins).

I may try again with a google cloud VM running Windows Server 2016 (as they are free for 3 months) but errors like these are generally very difficult to debug. Good luck!

Thanks

• Steve

On Apr 1, 2018, at 5:07 AM, omrisk notifications@github.com wrote:

@Bilal-S , I've removed the empty settings (we have some machines running older versions of the connector and they needed the settings defined for some reason). I'll try your suggested configurations for a few days and post an update.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[Bilal-S]

@spaskovich thanks for trying it out. There are users that use the connector with ACF as well. However, I feel you on the issue. I can only hope we can find the source of the problem. Currently, Tomcat is throwing an error after attempting to allocate 88K of buffer space. Overflow error for buffer adding [88728] bytes at position [552]. Why the tomcat side believes that is needing to add 88K bytes of buffer I am not certain. This maybe fixed in later version of tomcat but I do not know until I have a good case for replication and try updgrades. @omrisk I recommend to attempt to use smaller buffer sizes to avoid the over-allocation of bytes, e.g start with default of 8096 and move up.

[spaskovich]

Thanks for the reply Bilal. Just FYI in case you see this in the future, I couldn’t pin down the amount of bytes tomcat was adding to the buffer to trigger this (that was the problem). The flow occurred like this, user logs into site (which naturally makes a series of database calls building a customer’s inventory (in this case savings bonds). We aren’t talking about anything that’s heavy or crazy. But it seemed like if the customer had OVER 80-100 bonds in their inventory...bang...the site would immediately blow up with these errors (even in my top frame, which just really makes admin-style calls for customer name, email address, etc). So basically I would see values different than 88k (basically less, definitely 100% of the time if there were more bonds).

My application is currently on Microsoft Access (I KNOW vomit but it works and it works well) on CF8.01 and it’s solid as a rock (using whatever version of IIS Microsoft Server 2008 R2 comes with - I can double check if you want).

Is it possible to use Lucee without the Boncode AJP connector or I think Lucee has to run Tomcat, this was my first venture into Lucee.

Thanks again and I hope you had a great weekend.

Thanks

• Steve

On Apr 1, 2018, at 10:30 PM, Bilal Soylu notifications@github.com wrote:

@spaskovich thanks for trying it out. There are users that use the connector with ACF as well. However, I feel you on the issue. I can only hope we can find the source of the problem. Currently, Tomcat is throwing an error after attempting to allocate 88K of buffer space. Overflow error for buffer adding [88728] bytes at position [552]. Why the tomcat side believes that is needing to add 88K bytes of buffer I am not certain. This maybe fixed in later version of tomcat but I do not know until I have a good case for replication and try updgrades. @omrisk I recommend to attempt to use smaller buffer sizes to avoid the over-allocation of bytes, e.g start with default of 8096 and move up.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[omrisk]

Just updating the the final byte size that worked for us was 65530. For some reason the BonCode started throwing access_violation exceptions some where in BonCodeIIS.BonCodeCallHandler.CheckExecution(System.Web.HttpContext) but it doesn't seem to be affecting any thing...

[Bilal-S]

Thank you for the update.

[Bilal-S]

closing issue.

[kd8ssq]

I know it's been a few years since this ticket was closed, but I'm experiencing the same issue. I'm actually seeing this on several servers . One server has an executionTimeout of 3 hours and another servers has a 30 min timeout. The odd thing is the Generic Communication Error seems to be triggered almost every 4 minutes exactly.

I've tried adding the PacketSize tag, increasing the connectionTimeout all with no success.

Hopefully there has been some progress.

Thanks, Adam

Generic Connector Communication Error:
Please check and adjust your setup:
Ensure that Tomcat is running on given host and port.
If this is a timeout error consider adjusting IIS timeout by changing executionTimeout attribute in web.config (see manual). [2021-12-07 12:06:24 ]

server.xml

<Connector protocol="AJP/1.3"
        port="9009"
        secret="secret"
        secretRequired="true"
        redirectPort="8443" 
         maxThreads="10000" 
         connectionTimeout="60000" />

BonCodeAJP13.settings

<Settings>
<Server>localhost</Server>
<Port>9009</Port>
<EnableRemoteAdmin>True</EnableRemoteAdmin>
<EnableHeaderDataSupport>True</EnableHeaderDataSupport>
<ForceSecureSession>False</ForceSecureSession>
<AllowEmptyHeaders>False</AllowEmptyHeaders>
<ModCFMLSecret>ModCFMLSecret</ModCFMLSecret>
<RequestSecret>RequestSecret</RequestSecret>
<FlushThresholdTicks>500000</FlushThresholdTicks>
<SkipIISCustomErrors>True</SkipIISCustomErrors>
</Settings>

[Bilal-S]

@kd8ssq

What version of connector are you using?

Normally when you receive this error there is an underlying cause that is hidden. Do you see any errors in catalina logs/ IIS logs? Is it a certain path that causes this error consistently?

[kd8ssq]

Based on the version history file, I'm using 1.0.41. If there's another way to find the version, let me know. It came with the latest version of lucee which was installed a month or so ago.

I don't see anything in the catalina log and the IIS log returns a 502 error.

Yes, it's a certain path I'm using to get this error. I have a script that scans pages of a website to check that hyperlinks on the page are valid. One specific page had several hundred links on it to be scanned and it threw the error when scanning that page. One interesting thing to note is that the scan saves its results in a structure and doesn't output anything till the script is complete. So during the time that the error happens, there's been no output for 4 minutes. If I change the script to output something for each iteration of the hyperlink scan, the script completes.

It seems to have some kind of issue with no output received in a set time period and it thinks something's wrong.

Thanks,

Adam

On Tue, Dec 7, 2021 at 9:15 PM Bilal Soylu @.***> wrote:

@kd8ssq https://github.com/kd8ssq

What version of connector are you using?

Normally when you receive this error there is an underlying cause that is hidden. Do you see any errors in catalina logs/ IIS logs? Is it a certain path that causes this error consistently?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Bilal-S/iis2tomcat/issues/63#issuecomment-988433421, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABQ3DW3F2QHFBXHGY45NRNLUP25SXANCNFSM4EVT2CXA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.