search

apache / pekko

Build highly concurrent, distributed, and resilient message-driven applications using Java/Scala
https://pekko.apache.org/
Apache License 2.0
1.17k stars 139 forks source link

FAILED: ClusterReceptionistSpec #164

Closed jrudolph closed 1 year ago

jrudolph commented 1 year ago

https://github.com/apache/incubator-pekko/actions/runs/4139596533/jobs/7157347234#step:6:26963

Uncaught error from thread [ClusterReceptionistSpec-13-pekko.actor.internal-dispatcher-10]: null, shutting down JVM since 'pekko.jvm-exit-on-fatal-error' is enabled for ActorSystem[ClusterReceptionistSpec-13]
java.lang.ExceptionInInitializerError
    at org.apache.pekko.cluster.ddata.protobuf.msg.ReplicatorMessages$OtherMessage.getDescriptor(ReplicatorMessages.java:21503)
    at org.apache.pekko.cluster.ddata.protobuf.msg.ReplicatorMessages$OtherMessage.hashCode(ReplicatorMessages.java:21658)
    at scala.runtime.Statics.anyHash(Statics.java:122)
    at scala.collection.immutable.HashMap.elemHashCode(HashMap.scala:104)
    at scala.collection.immutable.HashMap.computeHash(HashMap.scala:113)
    at scala.collection.immutable.HashMap.updated(HashMap.scala:79)
    at scala.collection.immutable.Map$Map4.updated(Map.scala:467)
    at org.apache.pekko.cluster.ddata.protobuf.ReplicatedDataSerializer.$anonfun$orsetToProtoImpl$1(ReplicatedDataSerializer.scala:488)
    at scala.collection.Iterator.foreach(Iterator.scala:943)
    at scala.collection.Iterator.foreach$(Iterator.scala:943)
    at scala.collection.AbstractIterator.foreach(Iterator.scala:1431)
    at org.apache.pekko.cluster.ddata.protobuf.ReplicatedDataSerializer.orsetToProtoImpl(ReplicatedDataSerializer.scala:479)
    at org.apache.pekko.cluster.ddata.protobuf.ReplicatedDataSerializer.orsetToProto(ReplicatedDataSerializer.scala:468)
    at org.apache.pekko.cluster.ddata.protobuf.ReplicatedDataSerializer.$anonfun$multimapToProto$2(ReplicatedDataSerializer.scala:954)
    at org.apache.pekko.cluster.ddata.protobuf.ReplicatedDataSerializer.$anonfun$getEntries$1(ReplicatedDataSerializer.scala:682)
    at org.apache.pekko.cluster.ddata.protobuf.ReplicatedDataSerializer.$anonfun$getEntries$1$adapted(ReplicatedDataSerializer.scala:677)
    at scala.collection.immutable.Map$Map1.foreach(Map.scala:193)
    at org.apache.pekko.cluster.ddata.protobuf.ReplicatedDataSerializer.getEntries(ReplicatedDataSerializer.scala:677)
    at org.apache.pekko.cluster.ddata.protobuf.ReplicatedDataSerializer.multimapToProto(ReplicatedDataSerializer.scala:954)
    at org.apache.pekko.cluster.ddata.protobuf.ReplicatedDataSerializer.toBinary(ReplicatedDataSerializer.scala:396)
    at org.apache.pekko.cluster.ddata.protobuf.SerializationSupport.buildOther$1(SerializationSupport.scala:156)
    at org.apache.pekko.cluster.ddata.protobuf.SerializationSupport.otherMessageToProto(SerializationSupport.scala:172)
    at org.apache.pekko.cluster.ddata.protobuf.SerializationSupport.otherMessageToProto$(SerializationSupport.scala:150)
    at org.apache.pekko.cluster.ddata.protobuf.ReplicatorMessageSerializer.otherMessageToProto(ReplicatorMessageSerializer.scala:161)
    at org.apache.pekko.cluster.ddata.protobuf.ReplicatorMessageSerializer.dataEnvelopeToProto(ReplicatorMessageSerializer.scala:512)
    at org.apache.pekko.cluster.ddata.protobuf.ReplicatorMessageSerializer.toBinary(ReplicatorMessageSerializer.scala:243)
    at org.apache.pekko.cluster.ddata.Replicator.digest(Replicator.scala:2009)
    at org.apache.pekko.cluster.ddata.Replicator.getDigest(Replicator.scala:1994)
    at org.apache.pekko.cluster.ddata.Replicator.$anonfun$gossipTo$1(Replicator.scala:2145)
    at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:286)
    at scala.collection.immutable.Map$Map1.foreach(Map.scala:193)
    at scala.collection.TraversableLike.map(TraversableLike.scala:286)
    at scala.collection.TraversableLike.map$(TraversableLike.scala:279)
    at scala.collection.AbstractTraversable.map(Traversable.scala:108)
    at org.apache.pekko.cluster.ddata.Replicator.gossipTo(Replicator.scala:2145)
    at org.apache.pekko.cluster.ddata.Replicator.$anonfun$receiveGossipTick$1(Replicator.scala:2137)
    at org.apache.pekko.cluster.ddata.Replicator.$anonfun$receiveGossipTick$1$adapted(Replicator.scala:2137)
    at scala.Option.foreach(Option.scala:407)
    at org.apache.pekko.cluster.ddata.Replicator.receiveGossipTick(Replicator.scala:2137)
    at org.apache.pekko.cluster.ddata.Replicator$$anonfun$4.applyOrElse(Replicator.scala:1675)
    at org.apache.pekko.actor.Actor.aroundReceive(Actor.scala:547)
    at org.apache.pekko.actor.Actor.aroundReceive$(Actor.scala:545)
    at org.apache.pekko.cluster.ddata.Replicator.aroundReceive(Replicator.scala:1533)
    at org.apache.pekko.actor.ActorCell.receiveMessage(ActorCell.scala:590)
    at org.apache.pekko.actor.ActorCell.invoke(ActorCell.scala:557)
    at org.apache.pekko.dispatch.Mailbox.processMailbox(Mailbox.scala:280)
    at org.apache.pekko.dispatch.Mailbox.run(Mailbox.scala:241)
    at org.apache.pekko.dispatch.Mailbox.exec(Mailbox.scala:253)
    at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
    at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
    at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
    at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)
Caused by: java.lang.IllegalArgumentException: Failed to parse protocol buffer descriptor for generated code.
    at org.apache.pekko.protobufv3.internal.Descriptors$FileDescriptor.internalBuildGeneratedFileFrom(Descriptors.java:412)
    at org.apache.pekko.cluster.ddata.protobuf.msg.ReplicatorMessages.<clinit>(ReplicatorMessages.java:23982)
    ... 52 more
Caused by: org.apache.pekko.protobufv3.internal.InvalidProtocolBufferException: Protocol message end-group tag did not match expected tag.
    at org.apache.pekko.protobufv3.internal.InvalidProtocolBufferException.invalidEndTag(InvalidProtocolBufferException.java:129)
    at org.apache.pekko.protobufv3.internal.CodedInputStream$ArrayDecoder.checkLastTagWas(CodedInputStream.java:636)
    at org.apache.pekko.protobufv3.internal.AbstractParser.parsePartialFrom(AbstractParser.java:160)
    at org.apache.pekko.protobufv3.internal.AbstractParser.parseFrom(AbstractParser.java:191)
    at org.apache.pekko.protobufv3.internal.AbstractParser.parseFrom(AbstractParser.java:203)
    at org.apache.pekko.protobufv3.internal.AbstractParser.parseFrom(AbstractParser.java:208)
    at org.apache.pekko.protobufv3.internal.AbstractParser.parseFrom(AbstractParser.java:48)
    at org.apache.pekko.protobufv3.internal.DescriptorProtos$FileDescriptorProto.parseFrom(DescriptorProtos.java:2297)
    at org.apache.pekko.protobufv3.internal.Descriptors$FileDescriptor.internalBuildGeneratedFileFrom(Descriptors.java:410)
    ... 53 more
jrudolph commented 1 year ago

This is caused by the protobuf message classes (like org.apache.pekko.cluster.ddata.protobuf.msg.ReplicatorMessages) have not been regenerated after the rename of packages etc in the corresponding protobuf files. Instead, it seems in some strings in the binary data encoded as strings in those classes some oversimple replacements of akka->pekko has happened.

That does not work as those generated message files contain a binary representation of the corresponding protobuf files which is now broken. Instead those message classes need to be regenerated from the protobuf files. Hopefully, the protobuf format itself is still compatible (I think so, as plain protobuf does not carry metadata tags for data, though special wrappers like Any may be more problematic).

mdedetrich commented 1 year ago

This is caused by the protobuf message classes (like org.apache.pekko.cluster.ddata.protobuf.msg.ReplicatorMessages) have not been regenerated after the rename of packages etc in the corresponding protobuf files. Instead, it seems in some strings in the binary data encoded as strings in those classes some oversimple replacements of akka->pekko has happened.

That does not work as those generated message files contain a binary representation of the corresponding protobuf files which is now broken. Instead those message classes need to be regenerated from the protobuf files. Hopefully, the protobuf format itself is still compatible (I think so, as plain protobuf does not carry metadata tags for data, though special wrappers like Any may be more problematic).

So I am responsible for this, when I did the package rename change in Pekko I also had to deal with the fact that classes generated from Protobuff also change (or more specifically the packages). I installed protobuff locally and regenerated the binary data in the `java.lang.String[] descriptorData fields but evidently in some places it was either missed or done incorrectly.

FWIW, I couldn't actually install the ancient version of protobuff that Pekko is tied to, so I had to use the latest one, regenerated the class with the newer protobuff and then manually copy the java.lang.String[] descriptorData into the existing source.

jrudolph commented 1 year ago

FWIW, I couldn't actually install the ancient version of protobuff that Pekko is tied to,

I just copied the protoc executable from https://github.com/protocolbuffers/protobuf/releases/download/v3.11.4/protoc-3.11.4-linux-x86_64.zip somewhere in the filesystem and adapted the protoc setting in sbt to make it work.

mdedetrich commented 1 year ago

I just copied the protoc executable from https://github.com/protocolbuffers/protobuf/releases/download/v3.11.4/protoc-3.11.4-linux-x86_64.zip somewhere in the filesystem and adapted the protoc setting in sbt to make it work.

My issue is that I am on an M1 and they didn't provide binaries for ARM at that point.