search

quarkusio / quarkus

Quarkus: Supersonic Subatomic Java.
https://quarkus.io
Apache License 2.0
13.73k stars 2.67k forks source link

Connection to Kafka DevServices not possible #22775

Closed twobiers closed 2 years ago

twobiers commented 2 years ago

Describe the bug

Quarkus cant connect to the auto-configured DevServices Kafka Broker.

Expected behavior

Quarkus should connect seamless

Actual behavior

Quarkus tries to connect, but cant establish connection.

2022-01-10 17:34:27,191 WARN  [org.apa.kaf.cli.NetworkClient] (kafka-producer-network-thread | kafka-producer-source-out) [Producer clientId=kafka-producer-source-out] Connection to node -1 (localhost/127.0.0.1:49156) could not be established. Broker may not be available.
2022-01-10 17:34:27,191 WARN  [org.apa.kaf.cli.NetworkClient] (kafka-producer-network-thread | kafka-producer-source-out) [Producer clientId=kafka-producer-source-out] Bootstrap broker localhost:49156 (id: -1 rack: null) disconnected
2022-01-10 17:34:27,362 WARN  [org.apa.kaf.cli.NetworkClient] (smallrye-kafka-consumer-thread-1) [Consumer clientId=kafka-consumer-source-in, groupId=code-with-quarkus] Connection to node -1 (localhost/127.0.0.1:49156) could not be established. Broker may not be available.
2022-01-10 17:34:27,362 WARN  [org.apa.kaf.cli.NetworkClient] (smallrye-kafka-consumer-thread-1) [Consumer clientId=kafka-consumer-source-in, groupId=code-with-quarkus] Bootstrap broker localhost:49156 (id: -1 rack: null) disconnected
2022-01-10 17:34:28,780 WARN  [org.apa.kaf.cli.NetworkClient] (kafka-producer-network-thread | kafka-producer-uppercase-out) [Producer clientId=kafka-producer-uppercase-out] Connection to node -1 (localhost/127.0.0.1:49156) could not be established. Broker may not be available.
2022-01-10 17:34:28,780 WARN  [org.apa.kaf.cli.NetworkClient] (kafka-producer-network-thread | kafka-producer-uppercase-out) [Producer clientId=kafka-producer-uppercase-out] Bootstrap broker localhost:49156 (id: -1 rack: null) disconnected
2022-01-10 17:34:28,826 WARN  [org.apa.kaf.cli.NetworkClient] (smallrye-kafka-consumer-thread-1) [Consumer clientId=kafka-consumer-source-in, groupId=code-with-quarkus] Connection to node -1 (localhost/127.0.0.1:49156) could not be established. Broker may not be available.
2022-01-10 17:34:28,826 WARN  [org.apa.kaf.cli.NetworkClient] (smallrye-kafka-consumer-thread-1) [Consumer clientId=kafka-consumer-source-in, groupId=code-with-quarkus] Bootstrap broker localhost:49156 (id: -1 rack: null) disconnected
2022-01-10 17:34:28,889 WARN  [org.apa.kaf.cli.NetworkClient] (smallrye-kafka-consumer-thread-0) [Consumer clientId=kafka-consumer-uppercase-in, groupId=code-with-quarkus] Connection to node -1 (localhost/127.0.0.1:49156) could not be established. Broker may not be available.
2022-01-10 17:34:28,889 WARN  [org.apa.kaf.cli.NetworkClient] (smallrye-kafka-consumer-thread-0) [Consumer clientId=kafka-consumer-uppercase-in, groupId=code-with-quarkus] Bootstrap broker localhost:49156 (id: -1 rack: null) disconnected
2022-01-10 17:34:28,919 WARN  [org.apa.kaf.cli.NetworkClient] (kafka-producer-network-thread | kafka-producer-source-out) [Producer clientId=kafka-producer-source-out] Connection to node -1 (localhost/127.0.0.1:49156) could not be established. Broker may not be available.
2022-01-10 17:34:28,920 WARN  [org.apa.kaf.cli.NetworkClient] (kafka-producer-network-thread | kafka-producer-source-out) [Producer clientId=kafka-producer-source-out] Bootstrap broker localhost:49156 (id: -1 rack: null) disconnected

How to Reproduce?

Steps to Reproduce:

  1. Go to https://code.quarkus.io/?e=smallrye-reactive-messaging-kafka&extension-search=origin:platform%20kafka
  2. Download generated project, extract it
  3. Run mvn quarkus:dev

Output of uname -a or ver

Microsoft Windows [Version 10.0.22000.376]

Output of java -version

openjdk version "17.0.1" 2021-10-19 OpenJDK Runtime Environment Temurin-17.0.1+12 (build 17.0.1+12) OpenJDK 64-Bit Server VM Temurin-17.0.1+12 (build 17.0.1+12, mixed mode, sharing)

GraalVM version (if different from Java)

No response

Quarkus version or git rev

2.6.1.Final

Build tool (ie. output of mvnw --version or gradlew --version)

Apache Maven 3.8.4 (9b656c72d54e5bacbed989b64718c159fe39b537) Maven home: C:\ProgramData\chocolatey\lib\maven\apache-maven-3.8.4 Java version: 17.0.1, vendor: Eclipse Adoptium, runtime: C:\Program Files\Eclipse Adoptium\jdk-17.0.1.12-hotspot Default locale: de_DE, platform encoding: Cp1252 OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"

Additional information

$ docker --version
Docker version 20.10.11, build dea9396
$ docker ps -a
CONTAINER ID   IMAGE                         COMMAND                  CREATED         STATUS         PORTS                               NAMES
43bb689b4b88   vectorized/redpanda:v21.5.5   "sh -c 'while [ ! -f…"   4 seconds ago   Up 5 seconds   9644/tcp, 0.0.0.0:49157->9092/tcp   distracted_sinoussi
Docker Inspect ```sh $ docker inspect 43bb689b4b88 [ { "Id": "43bb689b4b88f9a4b5ad56212683b6c9a3a291d73d7c71a14230b1b8bce924e7", "Created": "2022-01-10T16:41:03.347847731Z", "Path": "sh", "Args": [ "-c", "while [ ! -f /var/lib/redpanda/redpanda.sh ]; do sleep 0.1; done; /var/lib/redpanda/redpanda.sh" ], "State": { "Status": "running", "Running": true, "Paused": false, "Restarting": false, "OOMKilled": false, "Dead": false, "Pid": 7817, "ExitCode": 0, "Error": "", "StartedAt": "2022-01-10T16:41:04.849808178Z", "FinishedAt": "0001-01-01T00:00:00Z" }, "Image": "sha256:98ca2a6da0045c0fb3ffde6a98b354a303216e79cbd82c0341a994383aa73fcb", "ResolvConfPath": "/var/lib/docker/containers/43bb689b4b88f9a4b5ad56212683b6c9a3a291d73d7c71a14230b1b8bce924e7/resolv.conf", "HostnamePath": "/var/lib/docker/containers/43bb689b4b88f9a4b5ad56212683b6c9a3a291d73d7c71a14230b1b8bce924e7/hostname", "HostsPath": "/var/lib/docker/containers/43bb689b4b88f9a4b5ad56212683b6c9a3a291d73d7c71a14230b1b8bce924e7/hosts", "LogPath": "/var/lib/docker/containers/43bb689b4b88f9a4b5ad56212683b6c9a3a291d73d7c71a14230b1b8bce924e7/43bb689b4b88f9a4b5ad56212683b6c9a3a291d73d7c71a14230b1b8bce924e7-json.log", "Name": "/distracted_sinoussi", "RestartCount": 0, "Driver": "overlay2", "Platform": "linux", "MountLabel": "", "ProcessLabel": "", "AppArmorProfile": "", "ExecIDs": null, "HostConfig": { "Binds": [], "ContainerIDFile": "", "LogConfig": { "Type": "json-file", "Config": {} }, "NetworkMode": "571288e853ccf2cdab20ad617c290be140a587deb6a8fa2224316a6748cc09f7", "PortBindings": { "9092/tcp": [ { "HostIp": "", "HostPort": "0" } ] }, "RestartPolicy": { "Name": "", "MaximumRetryCount": 0 }, "AutoRemove": false, "VolumeDriver": "", "VolumesFrom": [], "CapAdd": null, "CapDrop": null, "CgroupnsMode": "host", "Dns": null, "DnsOptions": null, "DnsSearch": null, "ExtraHosts": [], "GroupAdd": null, "IpcMode": "shareable", "Cgroup": "", "Links": null, "OomScoreAdj": 0, "PidMode": "", "Privileged": false, "PublishAllPorts": false, "ReadonlyRootfs": false, "SecurityOpt": null, "UTSMode": "", "UsernsMode": "", "ShmSize": 67108864, "Runtime": "runc", "ConsoleSize": [ 0, 0 ], "Isolation": "", "CpuShares": 0, "Memory": 0, "NanoCpus": 0, "CgroupParent": "", "BlkioWeight": 0, "BlkioWeightDevice": null, "BlkioDeviceReadBps": null, "BlkioDeviceWriteBps": null, "BlkioDeviceReadIOps": null, "BlkioDeviceWriteIOps": null, "CpuPeriod": 0, "CpuQuota": 0, "CpuRealtimePeriod": 0, "CpuRealtimeRuntime": 0, "CpusetCpus": "", "CpusetMems": "", "Devices": null, "DeviceCgroupRules": null, "DeviceRequests": null, "KernelMemory": 0, "KernelMemoryTCP": 0, "MemoryReservation": 0, "MemorySwap": 0, "MemorySwappiness": null, "OomKillDisable": false, "PidsLimit": null, "Ulimits": null, "CpuCount": 0, "CpuPercent": 0, "IOMaximumIOps": 0, "IOMaximumBandwidth": 0, "MaskedPaths": [ "/proc/asound", "/proc/acpi", "/proc/kcore", "/proc/keys", "/proc/latency_stats", "/proc/timer_list", "/proc/timer_stats", "/proc/sched_debug", "/proc/scsi", "/sys/firmware" ], "ReadonlyPaths": [ "/proc/bus", "/proc/fs", "/proc/irq", "/proc/sys", "/proc/sysrq-trigger" ] }, "GraphDriver": { "Data": { "LowerDir": "/var/lib/docker/overlay2/7f42a7a2577b2c9cc2904a42af8506e745c0e5ce488938947413183d06a2fab5-init/diff:/var/lib/docker/overlay2/cb46326572a52dfdb21a8e8222d68c5f9b41b9242485e4d07ceb90d2e4553bfe/diff:/var/lib/docker/overlay2/2c551f41c6000a014134f217fdd60d6eb3119e2aa139f6be396108fad2246ce9/diff:/var/lib/docker/overlay2/f088d0ab6d0188c1e08bcb4f31d7928a707ffabfe73ce2326a67e8aad18157ef/diff:/var/lib/docker/overlay2/45d012e80025c8d2a215d56f16f62d8d6f2c4071f3e369308df98f4adcdc15a0/diff:/var/lib/docker/overlay2/7710b423cdb4c71f4c25753d8759cc141726058b1427979f292295890b59f7e5/diff:/var/lib/docker/overlay2/da3ca8da892bc123e231d2c3896a33a090f875d904b6ae74da24d38a8232bc4f/diff:/var/lib/docker/overlay2/81995f9f90ef2919e51ee0269b93e3ec875b3026c6989b86f3f96cc9a7e22709/diff", "MergedDir": "/var/lib/docker/overlay2/7f42a7a2577b2c9cc2904a42af8506e745c0e5ce488938947413183d06a2fab5/merged", "UpperDir": "/var/lib/docker/overlay2/7f42a7a2577b2c9cc2904a42af8506e745c0e5ce488938947413183d06a2fab5/diff", "WorkDir": "/var/lib/docker/overlay2/7f42a7a2577b2c9cc2904a42af8506e745c0e5ce488938947413183d06a2fab5/work" }, "Name": "overlay2" }, "Mounts": [ { "Type": "volume", "Name": "6905004583ec1013b84061d51386a843da6a25278a1acb9176200521327439cc", "Source": "/var/lib/docker/volumes/6905004583ec1013b84061d51386a843da6a25278a1acb9176200521327439cc/_data", "Destination": "/var/lib/redpanda/data", "Driver": "local", "Mode": "", "RW": true, "Propagation": "" } ], "Config": { "Hostname": "43bb689b4b88", "Domainname": "", "User": "redpanda", "AttachStdin": false, "AttachStdout": false, "AttachStderr": false, "ExposedPorts": { "9092/tcp": {}, "9644/tcp": {} }, "Tty": false, "OpenStdin": false, "StdinOnce": false, "Env": [ "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "DEBIAN_FRONTEND=noninteractive" ], "Cmd": [ "-c", "while [ ! -f /var/lib/redpanda/redpanda.sh ]; do sleep 0.1; done; /var/lib/redpanda/redpanda.sh" ], "Image": "docker.io/vectorized/redpanda:v21.5.5", "Volumes": { "/var/lib/redpanda/data": {} }, "WorkingDir": "", "Entrypoint": [ "sh" ], "OnBuild": null, "Labels": { "maintainer": "Vectorized.io", "org.testcontainers": "true", "org.testcontainers.sessionId": "8c989546-c621-4462-b5f7-89ffdc8c6cb6", "quarkus-dev-service-kafka": "kafka" } }, "NetworkSettings": { "Bridge": "", "SandboxID": "1a90e1562dd85d895f0050baa8ef82bf6692151bdced5e3bcf7b247d8138e156", "HairpinMode": false, "LinkLocalIPv6Address": "", "LinkLocalIPv6PrefixLen": 0, "Ports": { "9092/tcp": [ { "HostIp": "0.0.0.0", "HostPort": "49157" } ], "9644/tcp": null }, "SandboxKey": "/var/run/docker/netns/1a90e1562dd8", "SecondaryIPAddresses": null, "SecondaryIPv6Addresses": null, "EndpointID": "", "Gateway": "", "GlobalIPv6Address": "", "GlobalIPv6PrefixLen": 0, "IPAddress": "", "IPPrefixLen": 0, "IPv6Gateway": "", "MacAddress": "", "Networks": { "19faf7ba-b3ba-4ad9-bc89-077c3ab06722": { "IPAMConfig": null, "Links": null, "Aliases": [ "kafka-DfS94", "43bb689b4b88" ], "NetworkID": "571288e853ccf2cdab20ad617c290be140a587deb6a8fa2224316a6748cc09f7", "EndpointID": "2dac3c297eca6a824fff3af380968a0130458c6e29cf392342f6a0e0e60211f0", "Gateway": "172.22.0.1", "IPAddress": "172.22.0.2", "IPPrefixLen": 16, "IPv6Gateway": "", "GlobalIPv6Address": "", "GlobalIPv6PrefixLen": 0, "MacAddress": "02:42:ac:16:00:02", "DriverOpts": null } } } } ] ````

Starting a Redpanda Container with the following config myself and setting kafka.bootstrap.servers=localhost:9092 works.

version: '3.7'
services:
  redpanda:
    command:
      - redpanda
      - start
      - --smp
      - '1'
      - --reserve-memory
      - 0M
      - --overprovisioned
      - --node-id
      - '0'
      - --kafka-addr
      - PLAINTEXT://0.0.0.0:29092,OUTSIDE://0.0.0.0:9092
      - --advertise-kafka-addr
      - PLAINTEXT://redpanda:29092,OUTSIDE://localhost:9092
    image: docker.vectorized.io/vectorized/redpanda:v21.11.2
    container_name: redpanda
    ports:
      - "9092:9092"
      - "29092:29092"
quarkus-bot[bot] commented 2 years ago

/cc @cescoffier, @ozangunalp, @stuartwdouglas

cescoffier commented 2 years ago

Can you paste the log of the broker?

twobiers commented 2 years ago

Yeah, sure

$ docker logs 411b7ca06436
We'd love to hear about your experience with redpanda:
https://vectorized.io/feedback
Starting redpanda...
Running:
HOSTNAME=411b7ca06436 PWD=/ HOME=/var/lib/redpanda SHLVL=1 PATH=/opt/redpanda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin DEBIAN_FRONTEND=noninteractive /opt/redpanda/bin/redpanda redpanda --redpanda-cfg /etc/redpanda/redpanda.yaml --overprovisioned --lock-memory=false --smp=1 --memory=1G --reserve-memory=0M

Welcome to the Redpanda community!

Slack: https://vectorized.io/slack - is the main way the community interacts with one another in real time :)
Twitter: https://twitter.com/vectorizedio - come say hi!
Github Discussion: https://github.com/vectorizedio/redpanda/discussions - is preferred for longer, async, thoughtful discussions
GitHub Issues: https://github.com/vectorizedio/redpanda/issues - is reserved only for actual issues. Please use the GitHub for discussions.
Documentation: https://vectorized.io/docs/ - official docs site
Support: https://support.vectorized.io/ - to share private information with the production support vectorized team
Product Feedback: https://vectorized.io/feedback/ - let us know how we can improve your experience

INFO   redpanda::main - application.cc:120 - Redpanda 4d2c42c - 4d2c42c15a6aca758292cb92975a9b0c322c6426
INFO   redpanda::main - application.cc:128 - kernel=5.10.16.3-microsoft-standard-WSL2, nodename=411b7ca06436, machine=x86_64
INFO  2022-01-10 16:51:36,628 [shard 0] redpanda::main - application.cc:115 - System resources: { cpus: 1, available memory: 1024.000GiB, reserved memory: 0.000bytes}
INFO  2022-01-10 16:51:36,630 [shard 0] redpanda::main - application.cc:270 - Configuration:

config_file: /etc/redpanda/redpanda.yaml
node_uuid: m8g9EbXB1h79FvruyaeYqzUZqEC4ivcSd2csidgfuQbuPZmEr
pandaproxy: {}
redpanda:
  admin:
    - address: 0.0.0.0
      port: 9644
  advertised_kafka_api:
    - address: localhost
      name: outside
      port: 49159
  auto_create_topics_enabled: true
  data_directory: /var/lib/redpanda/data
  developer_mode: true
  kafka_api:
    - address: 0.0.0.0
      name: outside
      port: 9092
  node_id: 0
  rpc_server:
    address: 0.0.0.0
    port: 33145
  seed_servers: []
rpk:
  coredump_dir: /var/lib/redpanda/coredump
  enable_memory_locking: false
  enable_usage_stats: true
  overprovisioned: true
  tune_aio_events: false
  tune_clocksource: false
  tune_coredump: false
  tune_cpu: false
  tune_disk_irq: false
  tune_disk_nomerges: false
  tune_disk_scheduler: false
  tune_disk_write_cache: false
  tune_fstrim: false
  tune_network: false
  tune_swappiness: false
  tune_transparent_hugepages: false

INFO  2022-01-10 16:51:36,630 [shard 0] redpanda::main - application.cc:274 - Use `rpk config set <cfg> <value>` to change values below:
INFO  2022-01-10 16:51:36,630 [shard 0] redpanda::main - application.cc:279 - redpanda.advertised_kafka_api:{{outside:{host: localhost, port: 49159}}}  - Address of Kafka API published to the clients
INFO  2022-01-10 16:51:36,630 [shard 0] redpanda::main - application.cc:279 - redpanda.kafka_qdc_max_depth:100  - Maximum queue depth used in kafka queue depth control.
INFO  2022-01-10 16:51:36,630 [shard 0] redpanda::main - application.cc:279 - redpanda.kafka_qdc_min_depth:1    - Minimum queue depth used in kafka queue depth control.
INFO  2022-01-10 16:51:36,630 [shard 0] redpanda::main - application.cc:279 - redpanda.kafka_qdc_idle_depth:10  - Queue depth when idleness is detected in kafka queue depth control.
INFO  2022-01-10 16:51:36,630 [shard 0] redpanda::main - application.cc:279 - redpanda.kafka_qdc_max_latency_ms:80      - Max latency threshold for kafka queue depth control depth tracking.
INFO  2022-01-10 16:51:36,630 [shard 0] redpanda::main - application.cc:279 - redpanda.kafka_qdc_depth_alpha:0.8        - Smoothing factor for kafka queue depth control depth tracking.
INFO  2022-01-10 16:51:36,630 [shard 0] redpanda::main - application.cc:279 - redpanda.kafka_qdc_enable:0       - Enable kafka queue depth control.
INFO  2022-01-10 16:51:36,630 [shard 0] redpanda::main - application.cc:279 - redpanda.kafka_qdc_window_count:12        - Number of windows used in kafka queue depth control latency tracking.
INFO  2022-01-10 16:51:36,630 [shard 0] redpanda::main - application.cc:279 - redpanda.superusers:{}    - List of superuser usernames
INFO  2022-01-10 16:51:36,630 [shard 0] redpanda::main - application.cc:279 - redpanda.cloud_storage_trust_file:{nullopt}       - Path to certificate that should be used to validate server certificate during TLS handshake
INFO  2022-01-10 16:51:36,630 [shard 0] redpanda::main - application.cc:279 - redpanda.cloud_storage_api_endpoint_port:443      - TLS port override
INFO  2022-01-10 16:51:36,630 [shard 0] redpanda::main - application.cc:279 - redpanda.cloud_storage_disable_tls:0      - Disable TLS for all S3 connections
INFO  2022-01-10 16:51:36,630 [shard 0] redpanda::main - application.cc:279 - redpanda.cloud_storage_reconciliation_interval_ms:10000   - Interval at which the archival service runs reconciliation (ms)
INFO  2022-01-10 16:51:36,630 [shard 0] redpanda::main - application.cc:279 - redpanda.cloud_storage_api_endpoint:{nullopt}     - Optional API endpoint
INFO  2022-01-10 16:51:36,630 [shard 0] redpanda::main - application.cc:279 - redpanda.cloud_storage_secret_key:{nullopt}       - AWS secret key
INFO  2022-01-10 16:51:36,630 [shard 0] redpanda::main - application.cc:279 - redpanda.auto_create_topics_enabled:1     - Allow topic auto creation
INFO  2022-01-10 16:51:36,630 [shard 0] redpanda::main - application.cc:279 - redpanda.compaction_ctrl_backlog_size:{nullopt}   - target backlog size for compaction controller. if not set compaction target compaction backlog would be equal to
INFO  2022-01-10 16:51:36,630 [shard 0] redpanda::main - application.cc:279 - redpanda.compaction_ctrl_min_shares:10    - minimum number of IO and CPU shares that compaction process can use
INFO  2022-01-10 16:51:36,630 [shard 0] redpanda::main - application.cc:279 - redpanda.compaction_ctrl_i_coeff:0        - integral coefficient for compaction PID controller.
INFO  2022-01-10 16:51:36,630 [shard 0] redpanda::main - application.cc:279 - redpanda.kafka_qdc_depth_update_ms:7000   - Update frequency for kafka queue depth control.
INFO  2022-01-10 16:51:36,630 [shard 0] redpanda::main - application.cc:279 - redpanda.compaction_ctrl_p_coeff:-12.5    - proportional coefficient for compaction PID controller. This has to be negative since compaction backlog should decrease when number of compaction shares increases
INFO  2022-01-10 16:51:36,630 [shard 0] redpanda::main - application.cc:279 - redpanda.id_allocator_batch_size:1000     - Id allocator allocates messages in batches (each batch is a one log record) and then serves requests from memory without touching the log until the batch is exhausted.
INFO  2022-01-10 16:51:36,630 [shard 0] redpanda::main - application.cc:279 - redpanda.id_allocator_log_capacity:100    - Capacity of the id_allocator log in number of messages. Once it reached id_allocator_stm should compact the log.
INFO  2022-01-10 16:51:36,630 [shard 0] redpanda::main - application.cc:279 - redpanda.reclaim_max_size:4194304 - Maximum batch cache reclaim size
INFO  2022-01-10 16:51:36,630 [shard 0] redpanda::main - application.cc:279 - redpanda.raft_transfer_leader_recovery_timeout_ms:10000   - Timeout waiting for follower recovery when transferring leadership
INFO  2022-01-10 16:51:36,630 [shard 0] redpanda::main - application.cc:279 - redpanda.fetch_session_eviction_timeout_ms:60000  - Minimum time before which unused session will get evicted from sessions. Maximum time after which inactive session will be deleted is two time given configuration valuecache
INFO  2022-01-10 16:51:36,630 [shard 0] redpanda::main - application.cc:279 - redpanda.rm_sync_timeout_ms:2000  - Time to wait state catch up before rejecting a request
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.max_compacted_log_segment_size:5368709120        - Max compacted segment size after consolidation
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.kvstore_flush_interval:10        - Key-value store flush interval (ms)
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.release_cache_on_segment_roll:0  - Free cache when segments roll
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.retention_bytes:{nullopt}        - max bytes per partition on disk before triggering a compaction
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.join_retry_timeout_ms:5000       - Time between cluster join retries in milliseconds
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.max_kafka_throttle_delay_ms:60000        - Fail-safe maximum throttle delay on kafka requests
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.reclaim_stable_window:10000      - Length of time above which growth is reset
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.raft_timeout_now_timeout_ms:1000 - Timeout for a timeout now request
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.recovery_append_timeout_ms:5000  - Timeout for append entries requests issued while updating stale follower
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.replicate_append_timeout_ms:3000 - Timeout for append entries requests issued while replicating entries
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.log_segment_size:1073741824      - How large in bytes should each log segment be (default 1G)
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.disable_batch_cache:0    - Disable batch cache in log manager
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.election_timeout_ms:1500 - Election timeout expressed in milliseconds
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.data_directory:{data_directory="/var/lib/redpanda/data"} - Place where redpanda will keep the data
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.default_topic_partitions:1       - Default number of partitions per topic
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.wait_for_leader_timeout_ms:5000  - Timeout (ms) to wait for leadership in metadata cache
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.default_topic_replications:1     - Default replication factor for new topics
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.kafka_api:{{outside:{host: 0.0.0.0, port: 9092}}}        - Address and port of an interface to listen for Kafka API requests
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.group_topic_partitions:1 - Number of partitions in the internal group membership topic
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.log_compaction_interval_ms:10000 - How often do we trigger background compaction
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.cloud_storage_enabled:0  - Enable archival storage
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.delete_retention_ms:604800000    - delete segments older than this - default 1 week
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.compaction_ctrl_d_coeff:0.2      - derivative coefficient for compaction PID controller.
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.cloud_storage_bucket:{nullopt}   - AWS bucket that should be used to store data
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.enable_transactions:0    - Enable transactions
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.cloud_storage_access_key:{nullopt}       - AWS access key
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.transactional_id_expiration_ms:604800000 - Producer ids are expired once this time has elapsed after the last write with the given producer id.
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.fetch_max_bytes:57671680 - Maximum number of bytes returned in fetch request
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.enable_coproc:0  - Enable coprocessing mode
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.fetch_reads_debounce_timeout:1   - Time to wait for next read in fetch request when requested min bytes wasn't reached
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.log_compression_type:producer    - Default topic compression type
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.log_message_timestamp_type:CreateTime    - Default topic messages timestamp type
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.kafka_qdc_latency_alpha:0.002    - Smoothing parameter for kafka queue depth control latency tracking.
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.node_id:0        - Unique id identifying a node in the cluster
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.log_cleanup_policy:delete        - Default topic cleanup policy
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.raft_heartbeat_interval_ms:150   - Milliseconds for raft leader heartbeats
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.readers_cache_eviction_timeout_ms:30000  - Duration after which inactive readers will be evicted from cache
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.reclaim_min_size:131072  - Minimum batch cache reclaim size
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.cloud_storage_max_connections:20 - Max number of simultaneous uploads to S3
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.raft_replicate_batch_window_size:1048576 - Max size of requests cached for replication
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.coproc_max_ingest_bytes:655360   - Maximum amount of data to hold from input logs in memory
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.rpc_server_tls:{ enabled: 0 key/cert files: {nullopt} ca file: {nullopt} client_auth_required: 0 }   - TLS configuration for RPC server
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.alter_topic_cfg_timeout_ms:5000  - Time to wait for entries replication in controller log when executing alter configuration requst
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.quota_manager_gc_sec:30000       - Quota manager GC frequency in milliseconds
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.rm_violation_recovery_policy:violation_recovery_policy: 0        - Describes how to recover from an invariant violation happened on the partition level
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.enable_admin_api:1       - Enable the admin API
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.raft_io_timeout_ms:10000 - Raft I/O timeout
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.stm_snapshot_recovery_policy:violation_recovery_policy: 0        - Describes how to recover from an invariant violation happened during reading a stm snapshot
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.tm_violation_recovery_policy:violation_recovery_policy: 0        - Describes how to recover from an invariant violation happened on the transaction coordinator level
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.group_new_member_join_timeout:30000      - Timeout for new member joins
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.kafka_qdc_window_size_ms:1500    - Window size for kafka queue depth control latency tracking.
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.admin_api_tls:{} - TLS configuration for admin HTTP server
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.group_max_session_timeout_ms:300000      - The maximum allowed session timeout for registered consumers. Longer timeouts give consumers more time to process messages in between heartbeats at the cost of a longer time to detect failures. Default quota tracking window size in milliseconds
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.disable_metrics:0        - Disable registering metrics
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.enable_pid_file:1        - Enable pid file. You probably don't want to change this.
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.kafka_group_recovery_timeout_ms:30000    - Kafka group recovery timeout expressed in milliseconds
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.reclaim_growth_window:3000       - Length of time in which reclaim sizes grow
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.dashboard_dir:{nullopt}  - serve http dashboard on / url
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.group_min_session_timeout_ms:6000        - The minimum allowed session timeout for registered consumers. Shorter timeouts result in quicker failure detection at the cost of more frequent consumer heartbeating, which can overwhelm broker resources.
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.rack:{nullopt}   - Rack identifier
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.coproc_max_batch_size:32768      - Maximum amount of bytes to read from one topic read
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.metadata_dissemination_retry_delay_ms:320        - Delay before retry a topic lookup in a shard or other meta tables
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.seed_server_meta_topic_partitions:7      - Number of partitions in internal raft metadata topic
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.metadata_dissemination_interval_ms:3000  - Interaval for metadata dissemination batching
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.default_num_windows:10   - Default number of quota tracking windows
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.tm_sync_timeout_ms:2000  - Time to wait state catch up before rejecting a request
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.controller_backend_housekeeping_interval_ms:1000 - Interval between iterations of controller backend housekeeping loop
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.raft_heartbeat_timeout_ms:3000   - raft heartbeat RPC timeout
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.min_version:0    - minimum redpanda compat version
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.kafka_api_tls:{} - TLS configuration for Kafka API endpoint
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.enable_idempotence:0     - Enable idempotent producer
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.seed_servers:{}  - List of the seed servers used to join current cluster. If the seed_server list is empty the node will be a cluster root and it will form a new cluster
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.advertised_rpc_api:{nullopt}     - Address of RPC endpoint published to other cluster members
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.enable_sasl:0    - Enable SASL authentication for Kafka connections.
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.kvstore_max_segment_size:16777216        - Key-value maximum segment size (bytes)
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.group_initial_rebalance_delay:300        - Extra delay (ms) added to rebalance phase to wait for new members
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.coproc_supervisor_server:{host: 127.0.0.1, port: 43189}  - IpAddress and port for supervisor service
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.coproc_offset_flush_interval_ms:300000   - Interval for which all coprocessor offsets are flushed to disk
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.segment_appender_flush_timeout_ms:1000   - Maximum delay until buffered data is written
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.rpc_server:{host: 0.0.0.0, port: 33145}  - IpAddress and port for RPC server
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.admin_api_doc_dir:/usr/share/redpanda/admin-api-doc      - Admin API doc directory
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.developer_mode:1 - Skips most of the checks performed at startup, not recomended for production use
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.coproc_max_inflight_bytes:10485760       - Maximum amountt of inflight bytes when sending data to wasm engine
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.admin:{{:{host: 0.0.0.0, port: 9644}}}   - Address and port of admin server
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.compaction_ctrl_max_shares:1000  - maximum number of IO and CPU shares that compaction process can use
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.target_quota_byte_rate:2147483648        - Target quota byte rate (bytes per second) - 2GB default
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.create_topic_timeout_ms:2000     - Timeout (ms) to wait for new topic creation
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.metadata_dissemination_retries:30        - Number of attempts of looking up a topic's meta data like shard before failing a request
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.use_scheduling_groups:0  - Manage CPU scheduling
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.default_window_sec:1000  - Default quota tracking window size in milliseconds
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.compaction_ctrl_update_interval_ms:30000 -
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.max_version:1    - max redpanda compat version
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.cloud_storage_region:{nullopt}   - AWS region that houses the bucket used for storage
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - redpanda.compacted_log_segment_size:268435456     - How large in bytes should each compacted log segment be (default 256MiB)
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - pandaproxy.advertised_pandaproxy_api:{}   - Rest API address and port to publish to client
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - pandaproxy.api_doc_dir:/usr/share/redpanda/proxy-api-doc  - API doc directory
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - pandaproxy.pandaproxy_api_tls:{}  - TLS configuration for Pandaproxy api
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - pandaproxy.pandaproxy_api:{{:{host: 0.0.0.0, port: 8082}}}        - Rest API listen address and port
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - pandaproxy_client.sasl_mechanism: - The SASL mechanism to use when connecting
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - pandaproxy_client.consumer_heartbeat_interval_ms:500      - Interval (in milliseconds) for consumer heartbeats
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - pandaproxy_client.consumer_rebalance_timeout_ms:2000      - Timeout (in milliseconds) for consumer rebalance
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - pandaproxy_client.consumer_request_max_bytes:1048576      - Max bytes to fetch per request
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - pandaproxy_client.consumer_request_timeout_ms:100 - Interval (in milliseconds) for consumer request timeout
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - pandaproxy_client.scram_username: - Username to use for SCRAM authentication mechanisms
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - pandaproxy_client.produce_batch_delay_ms:100      - Delay (in milliseconds) to wait before sending batch
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - pandaproxy_client.produce_batch_size_bytes:1048576        - Number of bytes to batch before sending to broker
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - pandaproxy_client.consumer_session_timeout_ms:10000       - Timeout (in milliseconds) for consumer session
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - pandaproxy_client.produce_batch_record_count:1000 - Number of records to batch before sending to broker
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - pandaproxy_client.retry_base_backoff_ms:100       - Delay (in milliseconds) for initial retry backoff
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - pandaproxy_client.retries:5       - Number of times to retry a request to a broker
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - pandaproxy_client.scram_password: - Password to use for SCRAM authentication mechanisms
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - pandaproxy_client.broker_tls:{ enabled: 0 key/cert files: {nullopt} ca file: {nullopt} client_auth_required: 0 }      - TLS configuration for the brokers
INFO  2022-01-10 16:51:36,631 [shard 0] redpanda::main - application.cc:279 - pandaproxy_client.brokers:{{host: 0.0.0.0, port: 9092}}   - List of address and port of the brokers
INFO  2022-01-10 16:51:36,631 [shard 0] syschecks - Writing pid file "/var/lib/redpanda/data/pid.lock"
INFO  2022-01-10 16:51:36,635 [shard 0] storage - directories.h:33 - Checking `/var/lib/redpanda/data` for supported filesystems
ERROR 2022-01-10 16:51:36,636 [shard 0] syschecks - Path: `/var/lib/redpanda/data' is not on XFS. This is a non-supported setup. Expect poor performance.
INFO  2022-01-10 16:51:36,662 [shard 0] redpanda::main - application.cc:516 - Partition manager started
INFO  2022-01-10 16:51:36,673 [shard 0] cluster - raft0_utils.h:32 - Current node is cluster root
INFO  2022-01-10 16:51:36,683 [shard 0] storage - segment.cc:626 - Creating new segment /var/lib/redpanda/data/redpanda/kvstore/0_0/0-0-v1.log
INFO  2022-01-10 16:51:36,694 [shard 0] raft - [group_id:0, {redpanda/controller/0}] consensus.cc:860 - Starting
INFO  2022-01-10 16:51:36,718 [shard 0] raft - [group_id:0, {redpanda/controller/0}] consensus.cc:890 - Recovered, log offsets: {start_offset:-9223372036854775808, committed_offset:-9223372036854775808, committed_offset_term:-9223372036854775808, dirty_offset:-9223372036854775808, dirty_offset_term:-9223372036854775808, last_term_start_offset:-9223372036854775808}, term:0
INFO  2022-01-10 16:51:36,755 [shard 0] cluster - state_machine.cc:29 - Starting state machine
INFO  2022-01-10 16:51:36,755 [shard 0] cluster - members_manager.cc:58 - starting cluster::members_manager...
INFO  2022-01-10 16:51:36,755 [shard 0] redpanda::main - application.cc:882 - Started RPC server listening at {host: 0.0.0.0, port: 33145}
INFO  2022-01-10 16:51:36,755 [shard 0] redpanda::main - application.cc:935 - Started Kafka API server listening at {{outside:{host: 0.0.0.0, port: 9092}}}
INFO  2022-01-10 16:51:36,756 [shard 0] raft - [group_id:0, {redpanda/controller/0}] vote_stm.cc:244 - became the leader term:1
INFO  2022-01-10 16:51:36,756 [shard 0] storage - segment.cc:626 - Creating new segment /var/lib/redpanda/data/redpanda/controller/0_0/0-1-v1.log
INFO  2022-01-10 16:51:36,756 [shard 0] admin_api_server - admin_server.cc:86 - Started HTTP admin service listening at {{:{host: 0.0.0.0, port: 9644}}}
INFO  2022-01-10 16:51:36,757 [shard 0] redpanda::main - application.cc:791 - Started Pandaproxy listening at {{:{host: 0.0.0.0, port: 8082}}}
INFO  2022-01-10 16:51:36,757 [shard 0] redpanda::main - application.cc:806 - Successfully started Redpanda!
cescoffier commented 2 years ago

it might be a windows issue. I would need to setup a VM (would have to wait this weekend if @ozangunalp does not beat me on it).

twobiers commented 2 years ago

I think the problem was somehow on my end. Don't know why and what caused it, but as I started my computer today it worked like a charm.