search

cloudant-labs / clouseau

Expose Lucene features as an erlang-like node
Apache License 2.0
58 stars 32 forks source link

Upgrade Scala version to be compatible with JDK 8 #22

Open AlexanderKaraberov opened 5 years ago

AlexanderKaraberov commented 5 years ago

Currently specified version is <scala.version>2.9.1</scala.version> which is quite old and is not compatible with JDK 8:

[ERROR] error: error while loading AnnotatedElement, class file '/Library/Java/JavaVirtualMachines/jdk1.8.0_151.jdk/Contents/Home/jre/lib/rt.jar(java/lang/reflect/AnnotatedElement.class)' is broken
[INFO] (bad constant pool tag 18 at byte 76)
[ERROR] error: error while loading CharSequence, class file '/Library/Java/JavaVirtualMachines/jdk1.8.0_151.jdk/Contents/Home/jre/lib/rt.jar(java/lang/CharSequence.class)' is broken
[INFO] (bad constant pool tag 18 at byte 10)

I checked Cloudant Maven repo and it doesn't contain newer versions. Is it possible to upgrade Scala to at least 2.10.2 or 2.11 ?

jiangphcn commented 5 years ago

It is possible, but it is not just to modify one configurable entry. We need to modify scala-lang and its dependency, such as overlock, etc, and also consider to upgrade approach of logging and metrics.

Some preliminary work was done and the result is promising.

https://github.com/boundary/overlock/compare/master...jiangphcn:support-scala-2.11?expand=1 https://github.com/boundary/scalang/compare/master...cloudant:scalang-support-scala-2.11?expand=1 https://github.com/cloudant/clouseau/compare/support-scala-2.11?expand=1

arturog commented 4 years ago

I also need to bring clouseau to a recent JDK. Last I looked we needed to remove a logging library from boundary, update scalang and overlock (also from boundary) and get everything to build with latest JDK. IMO we must fork these dependencies. Are we the only ones using them? o_O

CouchDB has decided to move to FoundationDB: https://lists.apache.org/thread.html/99c1a51918818d23a0049883c1233c4b0e76c0a71fba37c37f572e61@%3Cdev.couchdb.apache.org%3E

I have a question for cloudant: When are we expecting to complete our move to CouchDB + FoundationDB combo? 1month, 1year, 5years, 10years? It would be great to have a rough idea. We also needed to update the version of Lucene, but, is it worth the effort? If FoundationDB is still some time away I would be happy to start working on clouseau / scalang.

jiangphcn commented 4 years ago

Hi @arturog To my understanding, CouchDB 3.0 still uses dreyfus + clouseau while CouchDB 4.0 will be based on CouchDB + FoundationDB combo. They will co-exist over times. If you plan to officially update the version of Lucene, it will give benefit to CouchDB 3.0. I am doing similar thing to update spidermonkey version. To me, I am glad to see that you start working on clouseau/scalang. I am able of assisting you because you can see above commits I worked on.

arturog commented 4 years ago

Hi @jiangphcn, thanks for the info, that's very useful information.

Some of this update work was already done a couple of years ago: https://github.com/dmunch/clouseau

I think we can cherry-pick some of the commits there and bring the forks from cloudant up to speed. Once scala and jdk are updated, then we can try updating Lucene.

arturog commented 4 years ago

Hello again @jiangphcn, some good news. I have successfully move forward overlock and scalang from cloudant/boundary to scala_2.13. If you could start looking at the code and give some feedback that would be great. I still have lots of deprecations and warnings but I feel I've made some progress. Most deprecations left are around Symbol so I'm not too worried about. Please have a look at the tests and if you can make them pass that would be a huge step forward.

Instructions:

mkdir /tmp/scalang_2.13
cd /tmp/scalang_2.13
git clone https://github.com/arturog/overlock.git
cd overlock
mvn install:install-file -Dfile=./target/overlock-scala_2.13-0.9.0-SNAPSHOT.jar -DpomFile=./pom.xml

cd /tmp/scalang_2.13
git clone https://github.com/arturog/scalang.git
git checkout scala-2-13
mvn verify

Let me know what you think. Next steps would be:

  1. Remove Symbol and all deprecations
  2. Rebase and cleanup the branches
  3. Set versions for scalang and overlock to 1.0.0-SNAPSHOT, ideally under cloudant (please confirm we're happy to do this). Bye bye boundary.
  4. Update clouseau
  5. (for me) Release on a new system I have, ideally in the next week or so :-O!

Once this is working 100% I'll start updating Lucene

jiangphcn commented 4 years ago

Great process! I will give comments after some check.

arturog commented 4 years ago

Great, thanks! :-) After looking at the code, we need to have a chat about the use of "Symbol" in scalang - I went ahead and removed it in the latest commit. I feel in this case it was the right thing to do, but again I'm not a Scala expert.

(...clouseau will follow)

arturog commented 4 years ago

Hello @jiangphcn, I have updated clouseau and I can run it with scala_2_13. But I am now stuck and really need some guidance. I have a couple of tests that don't pass but I think it's a "Scala" thing - the code seems to be OK.

I am now stuck and really need a bit of guidance to get this done. Can you give it a quick test on your end?

https://github.com/arturog/clouseau

Instructions (after installing overlock and scalang above)

cd /tmp
git clone https://github.com/arturog/clouseau
cd clouseau
git checkout scala-2-13
mvn scala:run

Thanks!

arturog commented 4 years ago

I reverted the removal of "Symbol", I misunderstood that we were translating them to Erlang terms. The three branches above (overlock, scalang and clouseau) are now working with couchdb-master as of today. Also, I pushed --force (and will do so again).

arturog commented 4 years ago

OK, so where do we stand. I have updated dependencies to get couchdb-master + clouseu + dreyfus working with scala 2.13 and newer JDK. I have a few questions and comments though, it would be great to get some information on the below:

Clouseau

=> Main question: What commit is the working, stable version of clouseau that I should branch from to apply all scala-2.13 and Lucene upgrades?

Dreyfus

=> Main question: What commit of dreyfus should we use?

CouchDB => Main question: What commit is the stable couchdb to work with all the above?

I personally want to stick to a reliable couchdb-dreyfus-clouseau combo as I myself run patched versions of CouchDB-2.3.1. I would be happy to move forward our patches to master but without tags and knowing what is stable we are in the dark. CouchDB-2.3.1 removed some functionality we used (starting processes via the config), which was a bit annoying. We shrugged it off and moved to couchdb-2.3.1 but I feel we are again at square one branching from the "stable" couchdb-2.3.1. Also, looking at commits and future plans, CouchDB seems very keen on dropping features. Committing to "stable" releases seems to leave you on a path that is already deprecated (what happened from 2.2 to 2.3).

Thanks!

Arturo

jiangphcn commented 4 years ago

@arturog, my original plan is to give a try over weekend due to other project commitment recently. To your questions, my general answer is that the master in every repository is preferred base branch to continue.

Right now, we are using

above is being applied to cloudant.

Regarding to your questions:

[Peng Hui]: we should consider master

Is cloudant using the https://github.com/cloudant-labs/clouseau repo for their own deployments? do I have the latest code?

[Peng Hui]: yes, please use this repository to get latest codes

Clouseau refers to partitioned dbs, which makes me think it is expecting a couchdb running from master. Can someone clarify what commit is stable for the combo couchdb-dreyfus-clouseau?

[Peng Hui]: j8013 in cloudant is using 6c35acd8af3dad40f932ca179dbcbf0d5428498e from apache/couchdb to get codes for dreyfys. The clouseau is master in cloudant-labs/clouseau

=> Main question: What commit is the working, stable version of clouseau that I should branch from to apply all scala-2.13 and Lucene upgrades?

[Peng Hui]: in general, we should start from master plus the change for scalang

Dreyfus

Dreyfus master clearly doesn't apply to stable couchdb-2.3.1. What commit of dreyfus corresponds to stable-couchdb-2.3.1? Dreyfus has seen many patches in the past year, but they are all incompatible with couchdb-2.3.1. Are these expecting master to be used? Dreyfus has a branch 'dreyfus-new-and-old-clouseau'. Is clouseau from github "old-clouseau"? This one left me a bit astonished at first... Is there another clouseau?

=> Main question: What commit of dreyfus should we use? => Main question: What commit is the stable couchdb to work with all the above?

[Peng Hui]: please consider https://github.com/apache/couchdb/tree/master/src/dreyfus CouchDB

curlycabbage commented 4 years ago

@arturog, if there's anything this java noob can do to help, please let me know. I am trying to get clouseau running on a raspberry pi cluster, which is quite challenging with the java 1.6 dependency.

I have tried following your build steps, starting with overlock, and I am stuck on this error:

mvn install:install-file -Dfile=./target/overlock-scala_2.13-0.9.0-SNAPSHOT.jar -DpomFile=./pom.xml
... [snip] ....
[ERROR] The specified file '/tmp/scalang_2.13/overlock/target/overlock-scala_2.13-0.9.0-SNAPSHOT.jar' not exists

If you enable issues in your overlock or clouseau repos, I'll repost it there so we don't add more noise to this thread. Thanks!

arturog commented 4 years ago

Hi, basically you need to build overlock first, add it to your local repo and then build closeau. I am away from my laptop at the moment but will post some.more help. Cheers A.

On 23 November 2019 17:55:54 GMT+00:00, Peter Radocchia notifications@github.com wrote:

@arturog, if there's anything this java noob can do to help, please let me know. I am trying to get clouseau running on a raspberry pi cluster, which is quite challenging with the java 1.6 dependency.

I have tried following your build steps, starting with overlock, and I am stuck on this error:

mvn install:install-file
-Dfile=./target/overlock-scala_2.13-0.9.0-SNAPSHOT.jar
-DpomFile=./pom.xml
... [snip] ....
[ERROR] The specified file
'/tmp/scalang_2.13/overlock/target/overlock-scala_2.13-0.9.0-SNAPSHOT.jar'
not exists

If you enable issues your overlock or clouseau repos, I'll repost it there so we don't add more noise to this thread. Thanks!

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/cloudant-labs/clouseau/issues/22#issuecomment-557819126

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

arturog commented 4 years ago

@curlycabbage we can bring your issue over here: https://github.com/arturog/scalang/issues/1

arturog commented 4 years ago

Hello @jiangphcn:

I have made progress on the move to newer JDKs, specially with deprecations in Scala-2.13 and scalang. I feel I am ready to give it a good push and hopefully get this work over to you guys at cloudant for it to be merged in your repos. Please have a look at:

One issue that took me a very very long time is scalang and UTF-8 node names. For that I would like to loop in @rnewson as the original author of clouseau. Basically, there was a function called "fastString" in scalang that decoded erlang terms to Java. The function, in my opinion, could not handle UTF8 and consistently failed in the "ping" tests.

So I have removed it and all my tests now pass save one "monitor" one. If someone could have a good look at commit https://github.com/arturog/scalang/commit/1d0e3e5a9fae3ae681816da718d49f5e28801ff8 that would be great. I don't know if this change will slow things down in big deployments.

I really want to get this done to a point where I can get my patches to your repos. We will definitely use this new version of clouseau in one of our live deployments, but it would be very sad if these patches don't make it to your repos and stay in mine. Also, I'm happy to make "cloudant" changes if needed.

Thanks!!!

jiangphcn commented 4 years ago

Hi @arturog,

Great progress and thank you.

I can take a look at them during your Christmas season.

rnewson commented 4 years ago

My recollection is that fastString was a significant performance optimization (at the expense of tying itself to internal details of the JDK). Have you measured this difference?

arturog commented 4 years ago

No, I haven't measured the difference, though I expect it to be slower due to the decoding taking place.

I don't have much clarity right now on how get around this one. UTF-8 has to be decoded as it doesn't match unicode beyond position 127. On the bright side, almost 100% of the time erlang terms are in ASCII. Maybe a feature flag to tell scalang to use / not use fastString? I tried to fix fastString but I couldn't. Maybe someone with more Java skills can give it a go. I am using erts-10.5.1 / OTP22.

I think we should move forward with the three libraries and fix fastString in isolation. What do you think?

rnewson commented 4 years ago

Agreed, the library updates (excluding the fastString) bits seem uncontroversial, though I think for the stability of the project we'll need to mirror them under cloudant-labs.

rnewson commented 4 years ago

I've forked scalang and overlock to cloudant-labs, PR's to update those to JDK 8 compatibility would be a good step forward.

arturog commented 4 years ago

Great, let us clean-up my branches a bit more, do a bit of a rebase, and then I can send some PRs. Target Scala is 2.13 btw, what is cloudant aiming to use?

rnewson commented 4 years ago

Cloudant's plan is to migrate over to the https://github.com/cloudant-labs/search3-java and https://github.com/cloudant-labs/search3-erl projects with CouchDB 4.0.

arturog commented 4 years ago

Very nice. I think Scala-2.13 would be the better choice. It would give a good couple of years before it is fully replaced by search3, if not more. Thanks!

rnewson commented 4 years ago

agreed on 2.13.

jiangphcn commented 4 years ago

@arturog, I gave a try and in general search is working again with JDK 8.

Just several observations, 

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running scalang.NodeSpec
[info] NodeSpec
[info]
[info] Node should
[executor:2] INFO scalang.node.ErlangHandler - channel disconnected org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext@1575a693 [id: 0x52b6e6e6, /127.0.0.1:49345 :> /127.0.0.1:49340] DISCONNECTED. peer: 'tmp@localhost
[info]   + get connections from a remote node
[executor:5] INFO scalang.node.ErlangHandler - channel disconnected org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext@4a8217b5 [id: 0x427fd833, /127.0.0.1:49351 :> localhost/127.0.0.1:49348] DISCONNECTED. peer: 'test@localhost
[info]   + connect to a remote node
[executor:7] INFO scalang.node.ErlangHandler - channel disconnected org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext@7c45c088 [id: 0x6537a1aa, /127.0.0.1:49357 :> /127.0.0.1:49352] DISCONNECTED. peer: 'tmp@localhost
[info]   + accept pings
[executor:9] INFO scalang.node.ErlangHandler - channel disconnected org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext@7292a4c5 [id: 0x4060211c, /127.0.0.1:49363 :> localhost/127.0.0.1:49360] DISCONNECTED. peer: 'test@localhost
[info]   + send pings
[specs2-10] WARN scalang.ErlangNode - trouble sending message to 'taco_truck@localhost
[info]   + invalid pings should fail
[info]   + send local regname
[executor:8] INFO scalang.node.ErlangHandler - channel disconnected org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext@36cd5bf3 [id: 0x7aa98fb0, /127.0.0.1:49375 :> localhost/127.0.0.1:49372] DISCONNECTED. peer: 'test@localhost
[info]   + send remote regname
[executor:8] INFO scalang.node.ErlangHandler - channel disconnected org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext@55d171e3 [id: 0x2e5dba43, /127.0.0.1:49381 :> localhost/127.0.0.1:49378] DISCONNECTED. peer: 'test@localhost
[info]   + receive remove regname
[info]   + remove processes on exit
[info]   + deliver local breakages
[executor:7] INFO scalang.node.ErlangHandler - channel disconnected org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext@12bfacbe [id: 0x1ce8675f, /127.0.0.1:49392 :> /127.0.0.1:49387] DISCONNECTED. peer: 'test@localhost
[info]   + deliver remote breakages
[executor:6] INFO scalang.node.ErlangHandler - channel disconnected org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext@247c3e68 [id: 0x2cbcd5f4, /127.0.0.1:49398 :> /127.0.0.1:49393] DISCONNECTED. peer: 'test@localhost
[executor:7] ERROR scalang.node.ErlangHandler - error caught in erlang handler 'test@localhost
java.nio.channels.ClosedChannelException
    at org.jboss.netty.channel.socket.nio.NioWorker.cleanUpWriteBuffer(NioWorker.java:645)
    at org.jboss.netty.channel.socket.nio.NioWorker.writeFromUserCode(NioWorker.java:372)
    at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.handleAcceptedSocket(NioServerSocketPipelineSink.java:137)
    at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.eventSunk(NioServerSocketPipelineSink.java:76)
    at scalang.node.PacketCounter.writeRequested(PacketCounter.scala:30)
    at org.jboss.netty.handler.execution.ExecutionHandler.handleDownstream(ExecutionHandler.java:167)
    at org.jboss.netty.channel.Channels.write(Channels.java:632)
    at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEncoder.java:70)
    at scalang.node.PacketCounter.writeRequested(PacketCounter.scala:30)
    at org.jboss.netty.channel.Channels.write(Channels.java:632)
    at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEncoder.java:70)
    at scalang.node.PacketCounter.writeRequested(PacketCounter.scala:30)
    at org.jboss.netty.channel.Channels.write(Channels.java:611)
    at org.jboss.netty.channel.Channels.write(Channels.java:578)
    at org.jboss.netty.channel.AbstractChannel.write(AbstractChannel.java:251)
    at scalang.ErlangNode.getOrConnectAndSend(Node.scala:866)
    at scalang.ErlangNode.break(Node.scala:788)
    at scalang.node.LinkListenable.$anonfun$notifyBreak$1(LinkListenable.scala:29)
    at scalang.node.LinkListenable.$anonfun$notifyBreak$1$adapted(LinkListenable.scala:28)
    at scala.collection.immutable.List.foreach(List.scala:312)
    at scalang.node.LinkListenable.notifyBreak(LinkListenable.scala:28)
    at scalang.node.LinkListenable.notifyBreak$(LinkListenable.scala:27)
    at scalang.node.Link.notifyBreak(Link.scala:20)
    at scalang.node.Link.break(Link.scala:22)
    at scalang.node.ProcessAdapter.$anonfun$exit$2(ProcessAdapter.scala:125)
    at scalang.node.ProcessAdapter.$anonfun$exit$2$adapted(ProcessAdapter.scala:124)
    at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:576)
    at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:574)
    at scala.collection.AbstractIterable.foreach(Iterable.scala:904)
    at scalang.node.ProcessAdapter.exit(ProcessAdapter.scala:124)
    at scalang.node.ProcessAdapter.exit$(ProcessAdapter.scala:114)
    at scalang.node.MailboxProcess.exit(Mailbox.scala:33)
    at scalang.node.MailboxProcess.handleExit(Mailbox.scala:55)
    at scalang.ErlangNode.$anonfun$remoteBreak$1(Node.scala:771)
    at scalang.ErlangNode.$anonfun$remoteBreak$1$adapted(Node.scala:769)
    at scala.Option.foreach(Option.scala:438)
    at scalang.ErlangNode.remoteBreak(Node.scala:769)
    at scalang.node.ErlangHandler.messageReceived(ErlangHandler.scala:53)
    at scalang.node.FailureDetectionHandler.messageReceived(FailureDetectionHandler.scala:33)
    at scalang.node.PacketCounter.messageReceived(PacketCounter.scala:20)
    at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:302)
    at org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:76)
    at scalang.node.PacketCounter.messageReceived(PacketCounter.scala:20)
    at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:302)
    at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:321)
    at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:299)
    at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:216)
    at org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:69)
    at org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecutor$ChildExecutor.run(OrderedMemoryAwareThreadPoolExecutor.java:316)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
    at overlock.threadpool.ErrorLoggedThread.run(NamedThreadFactory.scala:40)
[info]   + deliver local breakages
discon
[executor:6] INFO scalang.node.ErlangHandler - channel disconnected org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext@1c6b16cf [id: 0x5cc8eb39, /127.0.0.1:49408 :> /127.0.0.1:49403] DISCONNECTED. peer: 'test@localhost
[info]   + deliver breaks on channel disconnect
[info]   + deliver local monitor exits
Failed tests:
  NodeSpec.$anonfun$new$77:169 'down -> 'noconnection != 'down -> 'blah expected:<('down,'[blah])> but was:<('down,'[noconnection])>

Tests run: 75, Failures: 1, Errors: 0, Skipped: 0
natcohen commented 4 years ago

@arturog and @jiangphcn CouchDB 3 is about to be released... do you think clouseau can be updated to latest scala/jdk/lucene soon?

arturog commented 4 years ago

@arturog and @jiangphcn CouchDB 3 is about to be released... do you think clouseau can be updated to latest scala/jdk/lucene soon?

I plan to get back into it this week. I'll post progress. -A.

jiangphcn commented 4 years ago

Cool, @arturog. Just be aware that CouchDB 4.0 will be based on different architecture/repository and doesn't need such update.

natcohen commented 4 years ago

@jiangphcn Any clue when 4.0 will be released? My guess is that it might take more than a year or two... In addition, CouchDB 3.0 will have Dreyfuss embedded which opens Clouseau to more users with a simplified installation.

Also @arturog seems almost done so I don't think it's a waste of time!

natcohen commented 4 years ago

@arturog CouchDB 3 is out :) any chance we can get an updated version of clouseau soon?

adcorduneanu commented 2 years ago

I keep reading about CouchDB 4.0, but 2 years passed since the last message, and there is no sign that it's ever coming... guess 3.X will be the last one of it.

J4RF commented 1 year ago

Any movement on this? Attempting to run clouseau 2.21.5 with couchdb 3.2.2 on macOS 12.5.1 using openjdk8 (1.8.0_362), clouseau emits the following error when couchdb is started:

[executor:21] WARN scalang.ErlangNode - Try to monitor non-live process: Pid('couchdb@1,1163,0,1677621701) -> 'net_k (Reference('couchdb@1,Vector(147483, 132907009, 562847734),1677621701))
[executor:21] ERROR scalang.node.ErlangHandler - error caught in erlang handler 'couchdb@127.0.0.1
java.nio.channels.UnresolvedAddressException
    at java.base/sun.nio.ch.Net.checkAddress(Net.java:149)
    at java.base/sun.nio.ch.Net.checkAddress(Net.java:157)
    at java.base/sun.nio.ch.SocketChannelImpl.checkRemote(SocketChannelImpl.java:753)
    at java.base/sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:774)
    at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink.connect(NioClientSocketPipelineSink.java:150)
    at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink.eventSunk(NioClientSocketPipelineSink.java:113)
    at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEncoder.java:60)
    at org.jboss.netty.channel.Channels.connect(Channels.java:541)
    at org.jboss.netty.channel.AbstractChannel.connect(AbstractChannel.java:210)
    at org.jboss.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:227)
    at org.jboss.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:188)
    at scalang.epmd.Epmd.<init>(Epmd.scala:58)
    at scalang.epmd.Epmd$.apply(Epmd.scala:33)
    at scalang.ErlangNode.connectAndSend(Node.scala:877)
    at scalang.ErlangNode$$anonfun$4.apply(Node.scala:859)
    at scalang.ErlangNode$$anonfun$4.apply(Node.scala:859)
    at overlock.atomicmap.OneShotThunk.value(OneShotThunk.scala:32)
    at overlock.atomicmap.OneShotThunk.equals(OneShotThunk.scala:35)
    at org.cliffc.high_scale_lib.NonBlockingHashMap.remove(NonBlockingHashMap.java:334)
    at overlock.atomicmap.AtomicMap.getOrElseUpdate(AtomicMap.scala:77)
    at scalang.ErlangNode.getOrConnectAndSend(Node.scala:858)
    at scalang.ErlangNode.monitorExit(Node.scala:631)
    at scalang.ErlangNode.monitorWithoutNotify(Node.scala:604)
    at scalang.node.ErlangHandler.messageReceived(ErlangHandler.scala:61)
    at scalang.node.FailureDetectionHandler.messageReceived(FailureDetectionHandler.scala:33)
    at scalang.node.PacketCounter.messageReceived(PacketCounter.scala:17)
    at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:302)
    at org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:76)
    at scalang.node.PacketCounter.messageReceived(PacketCounter.scala:17)
    at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:302)
    at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:321)
    at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:299)
    at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:216)
    at org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:69)
    at org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecutor$ChildExecutor.run(OrderedMemoryAwareThreadPoolExecutor.java:316)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
    at java.base/java.lang.Thread.run(Thread.java:832)
    at overlock.threadpool.ErrorLoggedThread.run(NamedThreadFactory.scala:40)
[executor:21] INFO scalang.node.ErlangHandler - channel disconnected org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext@1bb39cd3 [id: 0x7c7392c0, /127.0.0.1:62044 :> /127.0.0.1:61931] DISCONNECTED. peer: 'couchdb@127.0.0.1

Oddly enough, on my Ubuntu servers running the same version of clouseau (2.12.5) and openjdk8 (1.8.0_362), it is working with no issues. Initially tested with couchdb 3.1.1 on Ubuntu, then upgraded to 3.3.1. No issues on either version.

Anyone have any advice? Would love to get this working on my macOS dev environment.