search

fsprojects / pulsar-client-dotnet

Apache Pulsar native client for .NET (C#/F#/VB)
MIT License
303 stars 48 forks source link

Issues starting transactions #189

Closed tgrieger-sf closed 3 years ago

tgrieger-sf commented 3 years ago

I am using a Pulsar 2.8.0 container starting in standalone mode. I am following this page to enable transactions https://pulsar.apache.org/docs/en/next/txn-use/ (specifically, setting transactionCoordinatorEnabled to true and running bin/pulsar initialize-transaction-coordinator-metadata -cs 127.0.0.1:2181 -c standalone).

When I try to start a transaction with version 2.8.1, I get an exception Transaction manager is not started or not enabled with stack trace:

   at FSharp.Control.Tasks.TaskBuilder.bindTaskConfigureFalse@131.Invoke(Unit unitVar0)
   at FSharp.Control.Tasks.TaskBuilder.tryWith[a](FSharpFunc`2 step, FSharpFunc`2 catch)
--- End of stack trace from previous location ---
   at FSharp.Control.Tasks.TaskBuilder.bindTaskConfigureFalse@131.Invoke(Unit unitVar0)
   at FSharp.Control.Tasks.TaskBuilder.StepStateMachine`1.nextAwaitable()
--- End of stack trace from previous location ---
  ...

Here is a quick repro:

private async static Task TestPulsarContainer(string connectionString)
{
    string topicName = "testTopic";
    PulsarClient pulsarClient = await new PulsarClientBuilder().EnableTransaction(true).ServiceUrl(connectionString).BuildAsync();
    IProducer<byte[]> producer = await pulsarClient.NewProducer().Topic(topicName).CreateAsync();
    IConsumer<byte[]> consumer = await pulsarClient.NewConsumer().Topic(topicName).SubscriptionName("testSubscription").SubscribeAsync();

    string testMessage = "Test message";
    MessageId messageId = await producer.SendAsync(Encoding.UTF8.GetBytes(testMessage));
    Message<byte[]> message = await consumer.ReceiveAsync();

    // Exception occurs when creating the transaction.
    Transaction transaction = await pulsarClient.NewTransaction().BuildAsync();
    MessageBuilder<byte[]> messageBuilder = producer.NewMessage(Encoding.UTF8.GetBytes("test"), txn: transaction);
    await producer.SendAsync(messageBuilder);
    await consumer.AcknowledgeAsync(message.MessageId, transaction);
    await transaction.Commit();
}

Any ideas on if this is a config issue or a client issue?

Lanayx commented 3 years ago

Hi, it seems to be a misconfiguration problem, here's the relevant code https://github.com/apache/pulsar/blob/master/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ServerCnx.java#L1931. What is the output of the command bin/pulsar initialize-transaction-coordinator-metadata -cs 127.0.0.1:2181 -c standalone? Does it log successful start message? If it does the issue might happen because of connectivity issue - broker should be able to connect 2181 port to access transaction coordinator

tgrieger-sf commented 3 years ago

Here's the output of initializing the transaction coordinator.

bin/pulsar initialize-transaction-coordinator-metadata -cs 127.0.0.1:2181 -c standalone
14:06:10.981 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:zookeeper.version=3.6.3--6401e4ad2087061bc6b9f80dec2d69f2e3c8660a, built on 04/08/2021 16:35 GMT
14:06:10.983 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:host.name=PULSAR_1
14:06:10.984 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.version=11.0.11
14:06:10.984 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.vendor=Oracle Corporation
14:06:10.984 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.home=/usr/local/openjdk-11
14:06:10.984 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.class.path=/pulsar/conf:::/pulsar/lib/io.netty-netty-resolver-4.1.63.Final.jar:/pulsar/lib/io.swagger-swagger-annotations-1.6.2.jar:/pulsar/lib/commons-collections-commons-collections-3.2.2.jar:/pulsar/lib/org.apache.bookkeeper-stream-storage-server-4.14.1.jar:/pulsar/lib/io.netty-netty-buffer-4.1.63.Final.jar:/pulsar/lib/org.apache.pulsar-pulsar-package-core-2.8.0.jar:/pulsar/lib/com.google.protobuf-protobuf-java-3.11.4.jar:/pulsar/lib/org.apache.pulsar-pulsar-discovery-service-2.8.0.jar:/pulsar/lib/org.apache.pulsar-pulsar-functions-secrets-2.8.0.jar:/pulsar/lib/org.apache.bookkeeper-stream-storage-api-4.14.1.jar:/pulsar/lib/org.apache.logging.log4j-log4j-core-2.14.0.jar:/pulsar/lib/org.apache.pulsar-pulsar-client-admin-api-2.8.0.jar:/pulsar/lib/org.slf4j-jcl-over-slf4j-1.7.25.jar:/pulsar/lib/com.google.protobuf-protobuf-java-util-3.11.4.jar:/pulsar/lib/org.eclipse.jetty-jetty-client-9.4.42.v20210604.jar:/pulsar/lib/org.apache.zookeeper-zookeeper-jute-3.6.3.jar:/pulsar/lib/io.netty-netty-common-4.1.63.Final.jar:/pulsar/lib/io.swagger-swagger-models-1.6.2.jar:/pulsar/lib/com.fasterxml.jackson.module-jackson-module-jsonSchema-2.12.3.jar:/pulsar/lib/org.glassfish.hk2-osgi-resource-locator-1.0.3.jar:/pulsar/lib/org.apache.zookeeper-zookeeper-prometheus-metrics-3.6.3.jar:/pulsar/lib/org.slf4j-slf4j-api-1.7.25.jar:/pulsar/lib/io.prometheus-simpleclient_caffeine-0.5.0.jar:/pulsar/lib/com.yahoo.datasketches-sketches-core-0.8.3.jar:/pulsar/lib/com.github.ben-manes.caffeine-caffeine-2.6.2.jar:/pulsar/lib/javax.servlet-javax.servlet-api-3.1.0.jar:/pulsar/lib/org.apache.pulsar-pulsar-broker-2.8.0.jar:/pulsar/lib/org.apache.curator-curator-recipes-5.1.0.jar:/pulsar/lib/io.grpc-grpc-alts-1.33.0.jar:/pulsar/lib/org.apache.logging.log4j-log4j-web-2.14.0.jar:/pulsar/lib/commons-codec-commons-codec-1.10.jar:/pulsar/lib/org.eclipse.jetty-jetty-servlets-9.4.42.v20210604.jar:/pulsar/lib/org.apache.bookkeeper.http-vertx-http-server-4.14.1.jar:/pulsar/lib/org.bouncycastle-bcpkix-jdk15on-1.68.jar:/pulsar/lib/com.carrotsearch-hppc-0.7.3.jar:/pulsar/lib/org.apache.bookkeeper-stream-storage-proto-4.14.1.jar:/pulsar/lib/org.apache.pulsar-pulsar-websocket-2.8.0.jar:/pulsar/lib/io.prometheus-simpleclient_httpserver-0.5.0.jar:/pulsar/lib/com.fasterxml.jackson.core-jackson-core-2.12.3.jar:/pulsar/lib/org.apache.pulsar-pulsar-broker-auth-sasl-2.8.0.jar:/pulsar/lib/net.java.dev.jna-jna-4.2.0.jar:/pulsar/lib/org.eclipse.jetty-jetty-util-ajax-9.4.42.v20210604.jar:/pulsar/lib/org.apache.pulsar-pulsar-client-api-2.8.0.jar:/pulsar/lib/org.apache.pulsar-pulsar-common-2.8.0.jar:/pulsar/lib/org.glassfish.jersey.core-jersey-server-2.34.jar:/pulsar/lib/com.squareup.okhttp-okhttp-2.7.4.jar:/pulsar/lib/org.apache.commons-commons-collections4-4.1.jar:/pulsar/lib/io.prometheus-simpleclient_log4j2-0.5.0.jar:/pulsar/lib/org.apache.commons-commons-compress-1.19.jar:/pulsar/lib/javax.websocket-javax.websocket-client-api-1.0.jar:/pulsar/lib/org.xerial.snappy-snappy-java-1.1.7.jar:/pulsar/lib/io.grpc-grpc-stub-1.33.0.jar:/pulsar/lib/org.javassist-javassist-3.25.0-GA.jar:/pulsar/lib/org.apache.curator-curator-framework-5.1.0.jar:/pulsar/lib/org.apache.pulsar-pulsar-package-bookkeeper-storage-2.8.0.jar:/pulsar/lib/org.bitbucket.b_c-jose4j-0.7.6.jar:/pulsar/lib/org.eclipse.jetty.websocket-websocket-client-9.4.42.v20210604.jar:/pulsar/lib/io.kubernetes-client-java-proto-12.0.0.jar:/pulsar/lib/org.eclipse.jetty-jetty-server-9.4.42.v20210604.jar:/pulsar/lib/org.jctools-jctools-core-2.1.2.jar:/pulsar/lib/org.apache.avro-avro-protobuf-1.10.2.jar:/pulsar/lib/org.apache.httpcomponents-httpcore-4.4.13.jar:/pulsar/lib/org.apache.bookkeeper-bookkeeper-proto-4.14.1.jar:/pulsar/lib/org.apache.pulsar-pulsar-functions-local-runner-original-2.8.0.jar:/pulsar/lib/org.apache.pulsar-pulsar-transaction-coordinator-2.8.0.jar:/pulsar/lib/io.grpc-grpc-netty-1.33.0.jar:/pulsar/lib/io.airlift-aircompressor-0.16.jar:/pulsar/lib/io.grpc-grpc-xds-1.33.0.jar:/pulsar/lib/org.apache.pulsar-pulsar-proxy-2.8.0.jar:/pulsar/lib/org.apache.distributedlog-distributedlog-core-4.14.1-tests.jar:/pulsar/lib/commons-lang-commons-lang-2.6.jar:/pulsar/lib/com.fasterxml.jackson.core-jackson-databind-2.12.3.jar:/pulsar/lib/org.apache.bookkeeper-statelib-4.14.1.jar:/pulsar/lib/io.netty-netty-codec-socks-4.1.63.Final.jar:/pulsar/lib/org.eclipse.jetty-jetty-continuation-9.4.42.v20210604.jar:/pulsar/lib/com.google.guava-listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/pulsar/lib/org.apache.bookkeeper-bookkeeper-common-allocator-4.14.1.jar:/pulsar/lib/org.eclipse.jetty.websocket-websocket-server-9.4.42.v20210604.jar:/pulsar/lib/org.apache.pulsar-bouncy-castle-bc-2.8.0-pkg.jar:/pulsar/lib/org.apache.bookkeeper-stream-storage-common-4.14.1.jar:/pulsar/lib/io.kubernetes-client-java-api-12.0.0.jar:/pulsar/lib/io.netty-netty-transport-4.1.63.Final.jar:/pulsar/lib/org.apache.distributedlog-distributedlog-common-4.14.1.jar:/pulsar/lib/org.apache.distributedlog-distributedlog-protocol-4.14.1.jar:/pulsar/lib/com.google.errorprone-error_prone_annotations-2.5.1.jar:/pulsar/lib/org.apache.avro-avro-1.10.2.jar:/pulsar/lib/org.apache.pulsar-pulsar-testclient-2.8.0.jar:/pulsar/lib/org.glassfish.jersey.media-jersey-media-json-jackson-2.34.jar:/pulsar/lib/org.glassfish.jersey.core-jersey-client-2.34.jar:/pulsar/lib/io.prometheus-simpleclient_servlet-0.5.0.jar:/pulsar/lib/org.apache.bookkeeper-circe-checksum-4.14.1.jar:/pulsar/lib/com.google.re2j-re2j-1.2.jar:/pulsar/lib/io.opencensus-opencensus-proto-0.2.0.jar:/pulsar/lib/io.grpc-grpc-context-1.33.0.jar:/pulsar/lib/org.glassfish.jersey.inject-jersey-hk2-2.34.jar:/pulsar/lib/io.vertx-vertx-bridge-common-3.5.4.jar:/pulsar/lib/org.conscrypt-conscrypt-openjdk-uber-2.5.2.jar:/pulsar/lib/com.google.http-client-google-http-client-jackson2-1.34.0.jar:/pulsar/lib/commons-configuration-commons-configuration-1.10.jar:/pulsar/lib/org.apache.yetus-audience-annotations-0.5.0.jar:/pulsar/lib/org.glassfish.jersey.ext-jersey-entity-filtering-2.34.jar:/pulsar/lib/org.apache.curator-curator-client-5.1.0.jar:/pulsar/lib/org.apache.bookkeeper.http-http-server-4.14.1.jar:/pulsar/lib/com.google.auth-google-auth-library-oauth2-http-0.20.0.jar:/pulsar/lib/io.jsonwebtoken-jjwt-jackson-0.11.1.jar:/pulsar/lib/org.apache.logging.log4j-log4j-slf4j-impl-2.14.0.jar:/pulsar/lib/org.rocksdb-rocksdbjni-6.10.2.jar:/pulsar/lib/com.typesafe.netty-netty-reactive-streams-2.0.4.jar:/pulsar/lib/org.hdrhistogram-HdrHistogram-2.1.9.jar:/pulsar/lib/org.glassfish.hk2-hk2-utils-2.6.1.jar:/pulsar/lib/jakarta.validation-jakarta.validation-api-2.0.2.jar:/pulsar/lib/io.grpc-grpc-api-1.33.0.jar:/pulsar/lib/io.jsonwebtoken-jjwt-impl-0.11.1.jar:/pulsar/lib/io.dropwizard.metrics-metrics-core-3.2.5.jar:/pulsar/lib/org.apache.bookkeeper.stats-codahale-metrics-provider-4.14.1.jar:/pulsar/lib/org.eclipse.jetty-jetty-alpn-conscrypt-server-9.4.42.v20210604.jar:/pulsar/lib/org.glassfish.jersey.media-jersey-media-multipart-2.34.jar:/pulsar/lib/io.netty-netty-transport-native-unix-common-4.1.63.Final.jar:/pulsar/lib/io.jsonwebtoken-jjwt-api-0.11.1.jar:/pulsar/lib/io.vertx-vertx-web-3.5.4.jar:/pulsar/lib/io.grpc-grpc-core-1.33.0.jar:/pulsar/lib/org.glassfish.hk2-hk2-locator-2.6.1.jar:/pulsar/lib/io.grpc-grpc-protobuf-lite-1.33.0.jar:/pulsar/lib/io.opencensus-opencensus-contrib-http-util-0.24.0.jar:/pulsar/lib/org.apache.pulsar-pulsar-client-messagecrypto-bc-2.8.0.jar:/pulsar/lib/io.prometheus.jmx-collector-0.14.0.jar:/pulsar/lib/io.grpc-grpc-netty-shaded-1.33.0.jar:/pulsar/lib/io.prometheus-simpleclient_hotspot-0.5.0.jar:/pulsar/lib/com.google.j2objc-j2objc-annotations-1.3.jar:/pulsar/lib/com.yahoo.datasketches-memory-0.8.3.jar:/pulsar/lib/org.apache.pulsar-pulsar-transaction-common-2.8.0.jar:/pulsar/lib/io.perfmark-perfmark-api-0.19.0.jar:/pulsar/lib/io.vertx-vertx-auth-common-3.5.4.jar:/pulsar/lib/org.apache.pulsar-pulsar-config-validation-2.8.0.jar:/pulsar/lib/org.apache.pulsar-pulsar-functions-utils-2.8.0.jar:/pulsar/lib/org.yaml-snakeyaml-1.27.jar:/pulsar/lib/commons-logging-commons-logging-1.1.1.jar:/pulsar/lib/net.jcip-jcip-annotations-1.0.jar:/pulsar/lib/org.apache.bookkeeper-stream-storage-java-client-4.14.1.jar:/pulsar/lib/org.apache.pulsar-pulsar-client-admin-original-2.8.0.jar:/pulsar/lib/org.apache.distributedlog-distributedlog-core-4.14.1.jar:/pulsar/lib/io.netty-netty-codec-http2-4.1.63.Final.jar:/pulsar/lib/org.apache.zookeeper-zookeeper-3.6.3.jar:/pulsar/lib/io.gsonfire-gson-fire-1.8.5.jar:/pulsar/lib/org.apache.pulsar-pulsar-functions-instance-2.8.0.jar:/pulsar/lib/io.netty-netty-transport-native-unix-common-4.1.63.Final-linux-x86_64.jar:/pulsar/lib/com.beust-jcommander-1.78.jar:/pulsar/lib/org.apache.pulsar-pulsar-io-core-2.8.0.jar:/pulsar/lib/org.eclipse.jetty.websocket-websocket-common-9.4.42.v20210604.jar:/pulsar/lib/org.apache.bookkeeper-stream-storage-java-client-base-4.14.1.jar:/pulsar/lib/com.google.guava-failureaccess-1.0.1.jar:/pulsar/lib/com.fasterxml.jackson.jaxrs-jackson-jaxrs-base-2.12.3.jar:/pulsar/lib/io.opencensus-opencensus-api-0.18.0.jar:/pulsar/lib/org.eclipse.jetty.websocket-websocket-servlet-9.4.42.v20210604.jar:/pulsar/lib/io.dropwizard.metrics-metrics-jvm-3.2.5.jar:/pulsar/lib/org.eclipse.jetty-jetty-util-9.4.42.v20210604.jar:/pulsar/lib/com.sun.activation-javax.activation-1.2.0.jar:/pulsar/lib/io.prometheus-simpleclient-0.5.0.jar:/pulsar/lib/org.apache.pulsar-pulsar-metadata-2.8.0.jar:/pulsar/lib/io.vertx-vertx-core-3.5.4.jar:/pulsar/lib/com.google.code.findbugs-jsr305-3.0.2.jar:/pulsar/lib/io.netty-netty-codec-http-4.1.63.Final.jar:/pulsar/lib/org.apache.logging.log4j-log4j-api-2.14.0.jar:/pulsar/lib/org.apache.bookkeeper-stream-storage-service-api-4.14.1.jar:/pulsar/lib/org.asynchttpclient-async-http-client-2.12.1.jar:/pulsar/lib/javax.annotation-javax.annotation-api-1.3.2.jar:/pulsar/lib/org.apache.bookkeeper.stats-prometheus-metrics-provider-4.14.1.jar:/pulsar/lib/org.apache.bookkeeper-bookkeeper-tools-framework-4.14.1.jar:/pulsar/lib/io.netty-netty-codec-haproxy-4.1.63.Final.jar:/pulsar/lib/org.eclipse.jetty-jetty-http-9.4.42.v20210604.jar:/pulsar/lib/org.apache.pulsar-pulsar-functions-runtime-2.8.0.jar:/pulsar/lib/org.jvnet.mimepull-mimepull-1.9.13.jar:/pulsar/lib/org.apache.pulsar-pulsar-client-auth-sasl-2.8.0.jar:/pulsar/lib/org.glassfish.jersey.containers-jersey-container-servlet-2.34.jar:/pulsar/lib/org.apache.pulsar-pulsar-broker-common-2.8.0.jar:/pulsar/lib/org.apache.pulsar-testmocks-2.8.0.jar:/pulsar/lib/net.jodah-typetools-0.5.0.jar:/pulsar/lib/com.google.api.grpc-proto-google-common-protos-1.17.0.jar:/pulsar/lib/jakarta.activation-jakarta.activation-api-1.2.2.jar:/pulsar/lib/javax.xml.bind-jaxb-api-2.3.1.jar:/pulsar/lib/org.inferred-freebuilder-1.14.9.jar:/pulsar/lib/org.apache.thrift-libthrift-0.12.0.jar:/pulsar/lib/com.squareup.okhttp3-okhttp-3.14.9.jar:/pulsar/lib/io.grpc-grpc-protobuf-1.33.0.jar:/pulsar/lib/org.apache.pulsar-pulsar-functions-worker-2.8.0.jar:/pulsar/lib/io.netty-netty-codec-dns-4.1.63.Final.jar:/pulsar/lib/org.slf4j-jul-to-slf4j-1.7.25.jar:/pulsar/lib/io.netty-netty-resolver-dns-4.1.63.Final.jar:/pulsar/lib/org.eclipse.jetty.websocket-javax-websocket-client-impl-9.4.42.v20210604.jar:/pulsar/lib/org.apache.pulsar-pulsar-client-tools-2.8.0.jar:/pulsar/lib/org.glassfish.hk2-hk2-api-2.6.1.jar:/pulsar/lib/io.netty-netty-handler-proxy-4.1.63.Final.jar:/pulsar/lib/io.prometheus-simpleclient_jetty-0.5.0.jar:/pulsar/lib/org.apache.pulsar-pulsar-functions-api-2.8.0.jar:/pulsar/lib/com.squareup.okio-okio-1.13.0.jar:/pulsar/lib/com.google.guava-guava-30.1-jre.jar:/pulsar/lib/com.google.auth-google-auth-library-credentials-0.20.0.jar:/pulsar/lib/org.glassfish.jersey.containers-jersey-container-servlet-core-2.34.jar:/pulsar/lib/org.bouncycastle-bcprov-jdk15on-1.68.jar:/pulsar/lib/io.netty-netty-transport-native-epoll-4.1.63.Final-linux-x86_64.jar:/pulsar/lib/com.fasterxml.jackson.core-jackson-annotations-2.12.3.jar:/pulsar/lib/org.checkerframework-checker-qual-3.5.0.jar:/pulsar/lib/org.eclipse.jetty.websocket-websocket-api-9.4.42.v20210604.jar:/pulsar/lib/com.google.auto.value-auto-value-annotations-1.7.jar:/pulsar/lib/io.netty-netty-codec-4.1.63.Final.jar:/pulsar/lib/org.apache.pulsar-pulsar-functions-proto-2.8.0.jar:/pulsar/lib/io.grpc-grpc-okhttp-1.33.0.jar:/pulsar/lib/org.glassfish.hk2.external-jakarta.inject-2.6.1.jar:/pulsar/lib/org.eclipse.jetty-jetty-alpn-server-9.4.42.v20210604.jar:/pulsar/lib/com.fasterxml.jackson.dataformat-jackson-dataformat-yaml-2.12.3.jar:/pulsar/lib/javax.ws.rs-javax.ws.rs-api-2.1.jar:/pulsar/lib/org.apache.bookkeeper-cpu-affinity-4.14.1.jar:/pulsar/lib/org.glassfish.jersey.core-jersey-common-2.34.jar:/pulsar/lib/commons-io-commons-io-2.8.0.jar:/pulsar/lib/io.grpc-grpc-all-1.33.0.jar:/pulsar/lib/javax.validation-validation-api-1.1.0.Final.jar:/pulsar/lib/io.netty-netty-transport-native-epoll-4.1.63.Final.jar:/pulsar/lib/org.apache.pulsar-managed-ledger-2.8.0.jar:/pulsar/lib/org.bouncycastle-bcprov-ext-jdk15on-1.68.jar:/pulsar/lib/io.netty-netty-handler-4.1.63.Final.jar:/pulsar/lib/com.github.zafarkhaja-java-semver-0.9.0.jar:/pulsar/lib/jakarta.ws.rs-jakarta.ws.rs-api-2.1.6.jar:/pulsar/lib/com.fasterxml.jackson.module-jackson-module-jaxb-annotations-2.12.3.jar:/pulsar/lib/org.apache.httpcomponents-httpclient-4.5.13.jar:/pulsar/lib/io.grpc-grpc-auth-1.33.0.jar:/pulsar/lib/org.apache.bookkeeper.stats-bookkeeper-stats-api-4.14.1.jar:/pulsar/lib/org.eclipse.jetty-jetty-proxy-9.4.42.v20210604.jar:/pulsar/lib/org.apache.bookkeeper-stream-storage-service-impl-4.14.1.jar:/pulsar/lib/org.apache.commons-commons-lang3-3.11.jar:/pulsar/lib/commons-cli-commons-cli-1.2.jar:/pulsar/lib/org.eclipse.jetty-jetty-io-9.4.42.v20210604.jar:/pulsar/lib/org.glassfish.hk2.external-aopalliance-repackaged-2.6.1.jar:/pulsar/lib/org.apache.bookkeeper-bookkeeper-common-4.14.1.jar:/pulsar/lib/org.apache.logging.log4j-log4j-1.2-api-2.14.0.jar:/pulsar/lib/com.squareup.okhttp3-logging-interceptor-3.14.9.jar:/pulsar/lib/jakarta.xml.bind-jakarta.xml.bind-api-2.3.3.jar:/pulsar/lib/org.apache.bookkeeper-bookkeeper-server-4.14.1.jar:/pulsar/lib/com.google.code.gson-gson-2.8.6.jar:/pulsar/lib/org.apache.pulsar-pulsar-client-original-2.8.0.jar:/pulsar/lib/io.swagger-swagger-core-1.6.2.jar:/pulsar/lib/io.grpc-grpc-grpclb-1.33.0.jar:/pulsar/lib/io.grpc-grpc-services-1.33.0.jar:/pulsar/lib/org.eclipse.jetty-jetty-servlet-9.4.42.v20210604.jar:/pulsar/lib/jakarta.annotation-jakarta.annotation-api-1.3.5.jar:/pulsar/lib/io.prometheus-simpleclient_common-0.5.0.jar:/pulsar/lib/com.fasterxml.jackson.jaxrs-jackson-jaxrs-json-provider-2.12.3.jar:/pulsar/lib/io.dropwizard.metrics-metrics-graphite-3.2.5.jar:/pulsar/lib/org.apache.pulsar-pulsar-zookeeper-utils-2.8.0.jar:/pulsar/lib/io.kubernetes-client-java-12.0.0.jar:/pulsar/lib/com.google.http-client-google-http-client-1.34.0.jar:/pulsar/lib/org.reactivestreams-reactive-streams-1.0.3.jar:/pulsar/lib/io.netty-netty-tcnative-boringssl-static-2.0.38.Final.jar:/pulsar/lib/org.eclipse.jetty-jetty-security-9.4.42.v20210604.jar:/pulsar/lib/org.asynchttpclient-async-http-client-netty-utils-2.12.1.jar:
14:06:10.988 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.library.path=/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib
14:06:10.988 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.io.tmpdir=/tmp
14:06:10.988 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.compiler=<NA>
14:06:10.989 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.name=Linux
14:06:10.989 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.arch=amd64
14:06:10.989 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.version=5.10.16.3-microsoft-standard-WSL2
14:06:10.989 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:user.name=root
14:06:10.989 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:user.home=/root
14:06:10.989 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:user.dir=/pulsar
14:06:10.989 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.memory.free=1955MB
14:06:10.990 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.memory.max=2048MB
14:06:10.990 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.memory.total=2048MB
14:06:11.006 [main] INFO  org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=127.0.0.1:2181 sessionTimeout=30000 watcher=org.apache.bookkeeper.zookeeper.ZooKeeperWatcherBase@2ccca26f
14:06:11.012 [main] INFO  org.apache.zookeeper.common.X509Util - Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation
14:06:11.023 [main] INFO  org.apache.zookeeper.ClientCnxnSocket - jute.maxbuffer value is 10485760 Bytes
14:06:11.031 [main] INFO  org.apache.zookeeper.ClientCnxn - zookeeper.request.timeout value is 0. feature enabled=false
14:06:11.059 [main-SendThread(127.0.0.1:2181)] INFO  org.apache.zookeeper.ClientCnxn - Opening socket connection to server localhost/127.0.0.1:2181.
14:06:11.060 [main-SendThread(127.0.0.1:2181)] INFO  org.apache.zookeeper.ClientCnxn - SASL config status: Will not attempt to authenticate using SASL (unknown error)
14:06:11.068 [main-SendThread(127.0.0.1:2181)] INFO  org.apache.zookeeper.ClientCnxn - Socket connection established, initiating session, client: /127.0.0.1:33568, server: localhost/127.0.0.1:2181
14:06:11.080 [main-SendThread(127.0.0.1:2181)] INFO  org.apache.zookeeper.ClientCnxn - Session establishment complete on server localhost/127.0.0.1:2181, session id = 0x100002521f60012, negotiated timeout = 30000
14:06:11.084 [main-EventThread] INFO  org.apache.bookkeeper.zookeeper.ZooKeeperWatcherBase - ZooKeeper client is connected now.
14:06:11.340 [main-SendThread(127.0.0.1:2181)] WARN  org.apache.zookeeper.ClientCnxn - An exception was thrown while closing send thread for session 0x100002521f60012.
org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x100002521f60012, likely server has closed socket
        at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77) ~[org.apache.zookeeper-zookeeper-3.6.3.jar:3.6.3]
        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) ~[org.apache.zookeeper-zookeeper-3.6.3.jar:3.6.3]
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1290) [org.apache.zookeeper-zookeeper-3.6.3.jar:3.6.3]
14:06:11.459 [main-EventThread] INFO  org.apache.pulsar.metadata.impl.ZKSessionWatcher - Got ZK session watch event: WatchedEvent state:Closed type:None path:null
14:06:11.459 [main] INFO  org.apache.zookeeper.ZooKeeper - Session: 0x100002521f60012 closed
14:06:11.460 [main-EventThread] INFO  org.apache.zookeeper.ClientCnxn - EventThread shut down for session: 0x100002521f60012
Transaction coordinator metadata setup success
Lanayx commented 3 years ago

This log seems to be ok, can you inspect broker's log when the error happens?

tgrieger-sf commented 3 years ago

There are no messages happening in the output of the container (if that's the broker's log) when that exception occurs. There is output when the topics and subscriptions are created.

tgrieger-sf commented 3 years ago

I got the transactions working. Instead of updating the values and running the transaction initialization command, I run this script at startup:

#!/bin/sh
sed -i 's/transactionCoordinatorEnabled=false/transactionCoordinatorEnabled=true/g' /pulsar/conf/broker.conf
sed -i 's/transactionCoordinatorEnabled=false/transactionCoordinatorEnabled=true/g' /pulsar/conf/standalone.conf
/pulsar/bin/pulsar standalone

Thanks for looking into it, @Lanayx