search

clj-commons / manifold

A compatibility layer for event-driven abstractions
1.02k stars 106 forks source link

the stack size of some thread-factories are too small. #79

Closed tyano closed 8 years ago

tyano commented 8 years ago

wait-pool-promise and execute-pool-promise have their own thread-factory and the stack size is specified as 1e2(100). It is too small.

my program suddenly becomes unstable (StackOverflowError occurs in some cases) after upgrading 'aleph' to the newest 0.4.1 (which uses manifold 0.1.4), so I noticed this problem.

Changes to never assign a stack size for the thread-factories, so they will use JVM-default stack size, and users will be able to change it through a java option (-Xss).

In manifold 0.1.4, a bug where a :thread-factory option is ignored in a instrumented-executor function is fixed, then this problem suddenly be appeared. Before the bug-fix on 0.1.4, the thread-factories simplly never had been used, so this problem never occurred.

ztellman commented 8 years ago

Oh, only the wait-pool should have had that set. Which pool had the stack overflows?

tyano commented 8 years ago

Well, I checked both of wait-pool and execute-pool without assigning a stack size again one by one on my program, and found that this problem only depends on the execute-pool. so I changed my pull-request. Now only the stack size of execute-pool is changed. please see 'Files Changed'.

greglook commented 8 years ago

Can this be merged? I'm also getting burned by this right now, specifically with the AWS SDK, which seems to create a lot of frames:

java.lang.StackOverflowError
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
        at java.net.SocketInputStream.read(SocketInputStream.java:170)
        at java.net.SocketInputStream.read(SocketInputStream.java:141)
        at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
        at sun.security.ssl.InputRecord.read(InputRecord.java:503)
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
        at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930)
        at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
        at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139)
        at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155)
        at org.apache.http.impl.BHttpConnectionBase.fillInputBuffer(BHttpConnectionBase.java:346)
        at org.apache.http.impl.BHttpConnectionBase.isStale(BHttpConnectionBase.java:366)
        at org.apache.http.impl.conn.CPool.validate(CPool.java:69)
        at org.apache.http.impl.conn.CPool.validate(CPool.java:43)
        at org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:247)
        at org.apache.http.pool.AbstractConnPool.access$000(AbstractConnPool.java:64)
        at org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:192)
        at org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:185)
        at org.apache.http.pool.PoolEntryFuture.get(PoolEntryFuture.java:107)
        at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.java:276)
        at org.apache.http.impl.conn.PoolingHttpClientConnectionManager$1.get(PoolingHttpClientConnectionManager.java:263)
        at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.amazonaws.http.conn.ClientConnectionRequestFactory$Handler.invoke(ClientConnectionRequestFactory.java:70)
        at com.amazonaws.http.conn.$Proxy6.get(Unknown Source)
        at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:190)
        at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
        at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
        at com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72)
        at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:787)
        at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:630)
        at com.amazonaws.http.AmazonHttpClient.doExecute(AmazonHttpClient.java:405)
        at com.amazonaws.http.AmazonHttpClient.executeWithTimer(AmazonHttpClient.java:367)
        at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:318)
        at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3787)
        at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1399)
        at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1264)
        ...2 frames omitted...
        at clojure.lang.AFn.run(AFn.java:22)
        at io.aleph.dirigiste.Executor$3.run(Executor.java:308)
        at io.aleph.dirigiste.Executor$Worker$1.run(Executor.java:62)
        at manifold.executor$thread_factory$reify__3062$f__3063.invoke(executor.clj:36)
        at clojure.lang.AFn.run(AFn.java:22)
        at java.lang.Thread.run(Thread.java:745)
ztellman commented 8 years ago

Sure, I'll cut a new release this weekend On Fri, Jun 24, 2016 at 1:24 PM Greg Look notifications@github.com wrote:

Can this be merged? I'm also getting burned by this right now, specifically with the AWS SDK, which seems to create a lot of frames:

java.lang.StackOverflowError at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:170) at java.net.SocketInputStream.read(SocketInputStream.java:141) at sun.security.ssl.InputRecord.readFully(InputRecord.java:465) at sun.security.ssl.InputRecord.read(InputRecord.java:503) at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973) at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930) at sun.security.ssl.AppInputStream.read(AppInputStream.java:105) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.BHttpConnectionBase.fillInputBuffer(BHttpConnectionBase.java:346) at org.apache.http.impl.BHttpConnectionBase.isStale(BHttpConnectionBase.java:366) at org.apache.http.impl.conn.CPool.validate(CPool.java:69) at org.apache.http.impl.conn.CPool.validate(CPool.java:43) at org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:247) at org.apache.http.pool.AbstractConnPool.access$000(AbstractConnPool.java:64) at org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:192) at org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:185) at org.apache.http.pool.PoolEntryFuture.get(PoolEntryFuture.java:107) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.java:276) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager$1.get(PoolingHttpClientConnectionManager.java:263) at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.amazonaws.http.conn.ClientConnectionRequestFactory$Handler.invoke(ClientConnectionRequestFactory.java:70) at com.amazonaws.http.conn.$Proxy6.get(Unknown Source) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:190) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55) at com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72) at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:787) at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:630) at com.amazonaws.http.AmazonHttpClient.doExecute(AmazonHttpClient.java:405) at com.amazonaws.http.AmazonHttpClient.executeWithTimer(AmazonHttpClient.java:367) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:318) at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3787) at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1399) at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1264) ...2 frames omitted... at clojure.lang.AFn.run(AFn.java:22) at io.aleph.dirigiste.Executor$3.run(Executor.java:308) at io.aleph.dirigiste.Executor$Worker$1.run(Executor.java:62) at manifold.executor$thread_factory$reify3062$f3063.invoke(executor.clj:36) at clojure.lang.AFn.run(AFn.java:22) at java.lang.Thread.run(Thread.java:745)

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/ztellman/manifold/pull/79#issuecomment-228453181, or mute the thread https://github.com/notifications/unsubscribe/AAB6P8ytLg6NqJAn9C57i6OJR43kojsSks5qPD0IgaJpZM4IezpG .