search

txpipe / oura

The tail of Cardano
https://txpipe.github.io/oura
Apache License 2.0
257 stars 69 forks source link

"failed to fill whole buffer" when starting daemon #309

Closed PegasusPool closed 2 years ago

PegasusPool commented 2 years ago

I'm trying to setup Oura with Kafka and I got stuck. Oura version: 1.4.1 Here's my toml file:

[source]
type = "N2C"
address = ["Unix", "/opt/cardano/cnode/sockets/node.socket"]
magic = "mainnet"

[source.intersect]
type = "Origin"

[source.mapper]
include_transaction_details = true

[[filters]]
type = "Fingerprint"

[[filters]]
type = "Selection"

[filters.check]
predicate = "variant_in"
argument = ["Transaction"]

[sink]
type = "Kafka"
brokers = ["mybrokergoeshere:9092"]
topic = "cardano-events"

Startup using oura daemon

Error message:

ERROR: Error(
    Io(
        Error {
            kind: UnexpectedEof,
            message: "failed to fill whole buffer",
        },
    ),
    State {
        next_error: None,
        backtrace: None,
    },
)

Things work fine if I change the sink to Terminal or Stdout, it seems to be a problem with kafka. I also tried removing the filters and some source settings, but no luck.

My Kafka cluster is live and the topic is created.

When running with RUST_LOG=debug oura daemon the output is the following:

2022-06-02T12:50:12Z DEBUG oura::daemon] daemon starting with this config: ConfigRoot { source: N2C(Config { address: AddressArg(Unix, "/opt/cardano/cnode/sockets/node.socket"), magic: Some(MagicArg(764824073)), since: None, intersect: Some(Origin), well_known: None, mapper: Config { include_block_end_events: false, include_transaction_details: true, include_transaction_end_events: false, include_block_details: false, include_block_cbor: false, include_byron_ebb: false }, min_depth: 6, retry_policy: None }), filters: [Fingerprint(Config { seed: None }), Selection(Config { check: VariantIn(["Transaction"]) })], sink: Kafka(Config { brokers: ["xxx:9092"], topic: "cardano-events", ack_timeout_secs: None, paritioning: None }), chain: None, cursor: None, metrics: None }
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 0, ts: 4244906979, payload length: 10
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::setup] handshake output: Accepted(32778, VersionData(764824073))
[2022-06-02T12:50:12Z INFO  oura::sources::common] found 'origin' intersect argument, will use as starting point
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::setup] starting chain sync from: None
[2022-06-02T12:50:12Z DEBUG kafka::client] fetch_metadata: requesting metadata from xxx:9092
[2022-06-02T12:50:12Z WARN  pallas_multiplexer] protocol handle 0 disconnected
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244918655, payload length: 49
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_miniprotocols::chainsync::agents] rolling backward to point: Origin
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::run] rolling block to point Origin
[2022-06-02T12:50:12Z DEBUG oura::sources::n2c::run] rollback out of buffer scope, sending event down the pipeline
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::run] rollback buffer state, size: 0, oldest: None, latest: None
[2022-06-02T12:50:12Z DEBUG oura::filters::fingerprint] computed fingerprint 0.back.0
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244927831, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244927856, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244927870, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244927876, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244927882, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244927888, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244927894, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244927900, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244927905, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244927910, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244927917, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244927922, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244927928, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244927934, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244927940, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244927950, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244927959, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244927964, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244927970, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244927977, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244927983, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244927989, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244927995, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244928001, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244928009, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244928016, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244928022, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244928029, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244928035, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244928041, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244928047, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244928253, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244928268, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244928276, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244928286, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244928292, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244928299, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244928307, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244928316, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244928331, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244928347, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244928363, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244928372, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244928378, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244928384, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244928389, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244928395, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244928401, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244928407, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244928413, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244928419, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244928425, payload length: 12288
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244928444, payload length: 9166
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_miniprotocols::chainsync::agents] rolling forward
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::run] rolling forward to point (0, 89d9b5a5b8ddc8d7e5a6795e9774d97faf1efea59b2caf7eaf9f8c5b32059df4)
[2022-06-02T12:50:12Z DEBUG oura::sources::n2c::run] found 0 points with required min depth
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::run] rollback buffer state, size: 1, oldest: Some((0, 89d9b5a5b8ddc8d7e5a6795e9774d97faf1efea59b2caf7eaf9f8c5b32059df4)), latest: Some((0, 89d9b5a5b8ddc8d7e5a6795e9774d97faf1efea59b2caf7eaf9f8c5b32059df4))
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244937309, payload length: 720
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_miniprotocols::chainsync::agents] rolling forward
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::run] rolling forward to point (0, f0f7892b5c333cffc4b3c4344de48af4cc63f55e44936196f365a9ef2244134f)
[2022-06-02T12:50:12Z DEBUG oura::sources::n2c::run] found 0 points with required min depth
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::run] rollback buffer state, size: 2, oldest: Some((0, 89d9b5a5b8ddc8d7e5a6795e9774d97faf1efea59b2caf7eaf9f8c5b32059df4)), latest: Some((0, f0f7892b5c333cffc4b3c4344de48af4cc63f55e44936196f365a9ef2244134f))
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244947373, payload length: 720
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_miniprotocols::chainsync::agents] rolling forward
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::run] rolling forward to point (1, 1dbc81e3196ba4ab9dcb07e1c37bb28ae1c289c0707061f28b567c2f48698d50)
[2022-06-02T12:50:12Z DEBUG oura::sources::n2c::run] found 0 points with required min depth
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::run] rollback buffer state, size: 3, oldest: Some((0, 89d9b5a5b8ddc8d7e5a6795e9774d97faf1efea59b2caf7eaf9f8c5b32059df4)), latest: Some((1, 1dbc81e3196ba4ab9dcb07e1c37bb28ae1c289c0707061f28b567c2f48698d50))
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244957489, payload length: 720
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_miniprotocols::chainsync::agents] rolling forward
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::run] rolling forward to point (2, 52b7912de176ab76c233d6e08ccdece53ac1863c08cc59d3c5dec8d924d9b536)
[2022-06-02T12:50:12Z DEBUG oura::sources::n2c::run] found 0 points with required min depth
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::run] rollback buffer state, size: 4, oldest: Some((0, 89d9b5a5b8ddc8d7e5a6795e9774d97faf1efea59b2caf7eaf9f8c5b32059df4)), latest: Some((2, 52b7912de176ab76c233d6e08ccdece53ac1863c08cc59d3c5dec8d924d9b536))
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244967559, payload length: 720
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_miniprotocols::chainsync::agents] rolling forward
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::run] rolling forward to point (3, be06c81f4ad34d98578b67840d8e65b2aeb148469b290f6b5235e41b75d38572)
[2022-06-02T12:50:12Z DEBUG oura::sources::n2c::run] found 0 points with required min depth
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::run] rollback buffer state, size: 5, oldest: Some((0, 89d9b5a5b8ddc8d7e5a6795e9774d97faf1efea59b2caf7eaf9f8c5b32059df4)), latest: Some((3, be06c81f4ad34d98578b67840d8e65b2aeb148469b290f6b5235e41b75d38572))
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244977622, payload length: 720
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_miniprotocols::chainsync::agents] rolling forward
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::run] rolling forward to point (4, 46debe49b4fe0bc8c07cfe650de89632ca1ab5d58f04f8c88d8102da7ef79b7f)
[2022-06-02T12:50:12Z DEBUG oura::sources::n2c::run] found 0 points with required min depth
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::run] rollback buffer state, size: 6, oldest: Some((0, 89d9b5a5b8ddc8d7e5a6795e9774d97faf1efea59b2caf7eaf9f8c5b32059df4)), latest: Some((4, 46debe49b4fe0bc8c07cfe650de89632ca1ab5d58f04f8c88d8102da7ef79b7f))
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244987726, payload length: 720
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_miniprotocols::chainsync::agents] rolling forward
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::run] rolling forward to point (5, 365201e928da50760fce4bdad09a7338ba43a43aff1c0e8d3ec458388c932ec8)
[2022-06-02T12:50:12Z DEBUG oura::sources::n2c::run] found 1 points with required min depth
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::run] rollback buffer state, size: 6, oldest: Some((0, f0f7892b5c333cffc4b3c4344de48af4cc63f55e44936196f365a9ef2244134f)), latest: Some((5, 365201e928da50760fce4bdad09a7338ba43a43aff1c0e8d3ec458388c932ec8))
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4244997842, payload length: 720
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_miniprotocols::chainsync::agents] rolling forward
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::run] rolling forward to point (6, e39d988dd815fc2cb234c2abef0d7f57765eeffb67331814bdb01c590359325e)
[2022-06-02T12:50:12Z DEBUG oura::sources::n2c::run] found 1 points with required min depth
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::run] rollback buffer state, size: 6, oldest: Some((1, 1dbc81e3196ba4ab9dcb07e1c37bb28ae1c289c0707061f28b567c2f48698d50)), latest: Some((6, e39d988dd815fc2cb234c2abef0d7f57765eeffb67331814bdb01c590359325e))
[2022-06-02T12:50:12Z DEBUG oura::filters::fingerprint] computed fingerprint 0.blck.3181956016442758865248842286123015102
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4245007901, payload length: 720
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_miniprotocols::chainsync::agents] rolling forward
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::run] rolling forward to point (7, 2d9136c363c69ad07e1a918de2ff5aeeba4361e33b9c2597511874f211ca26e9)
[2022-06-02T12:50:12Z DEBUG oura::sources::n2c::run] found 1 points with required min depth
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::run] rollback buffer state, size: 6, oldest: Some((2, 52b7912de176ab76c233d6e08ccdece53ac1863c08cc59d3c5dec8d924d9b536)), latest: Some((7, 2d9136c363c69ad07e1a918de2ff5aeeba4361e33b9c2597511874f211ca26e9))
[2022-06-02T12:50:12Z DEBUG oura::filters::fingerprint] computed fingerprint 1.blck.32326382414103767099549474577546712059
[2022-06-02T12:50:12Z DEBUG kafka::client::network] Established: KafkaConnection { id: 0, secured: false, host: "xxx:9092" }
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4245017975, payload length: 720
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_miniprotocols::chainsync::agents] rolling forward
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::run] rolling forward to point (8, f4b2213e2ed6dc8d4da0f4234ced91714c8cbaa0f3750abb90d10aff23ce4d67)
[2022-06-02T12:50:12Z DEBUG oura::sources::n2c::run] found 1 points with required min depth
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::run] rollback buffer state, size: 6, oldest: Some((3, be06c81f4ad34d98578b67840d8e65b2aeb148469b290f6b5235e41b75d38572)), latest: Some((8, f4b2213e2ed6dc8d4da0f4234ced91714c8cbaa0f3750abb90d10aff23ce4d67))
[2022-06-02T12:50:12Z DEBUG oura::filters::fingerprint] computed fingerprint 2.blck.185427292007371968222431862127577108637
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4245028062, payload length: 720
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_miniprotocols::chainsync::agents] rolling forward
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::run] rolling forward to point (9, 9cf9488161d63c7c414008737e1fab2eb8b85bf55dfe3fd8a09d2bc0c4327fe3)
[2022-06-02T12:50:12Z DEBUG oura::sources::n2c::run] found 1 points with required min depth
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::run] rollback buffer state, size: 6, oldest: Some((4, 46debe49b4fe0bc8c07cfe650de89632ca1ab5d58f04f8c88d8102da7ef79b7f)), latest: Some((9, 9cf9488161d63c7c414008737e1fab2eb8b85bf55dfe3fd8a09d2bc0c4327fe3))
[2022-06-02T12:50:12Z DEBUG oura::filters::fingerprint] computed fingerprint 3.blck.280684594898126575256856968197803531607
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4245038138, payload length: 720
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_miniprotocols::chainsync::agents] rolling forward
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::run] rolling forward to point (10, 4bcd56b2e530a9690a94fb12a03cfd2cb7f25cdb229fda545c8cfddb57e13e24)
[2022-06-02T12:50:12Z DEBUG oura::sources::n2c::run] found 1 points with required min depth
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::run] rollback buffer state, size: 6, oldest: Some((5, 365201e928da50760fce4bdad09a7338ba43a43aff1c0e8d3ec458388c932ec8)), latest: Some((10, 4bcd56b2e530a9690a94fb12a03cfd2cb7f25cdb229fda545c8cfddb57e13e24))
[2022-06-02T12:50:12Z DEBUG oura::filters::fingerprint] computed fingerprint 4.blck.146041520578008116266575894971388760657
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4245048224, payload length: 720
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_miniprotocols::chainsync::agents] rolling forward
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::run] rolling forward to point (11, 7054258f4310b7803419681db10cd4a47707c5d5182ec9317fae7e2e93a399af)
[2022-06-02T12:50:12Z DEBUG oura::sources::n2c::run] found 1 points with required min depth
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::run] rollback buffer state, size: 6, oldest: Some((6, e39d988dd815fc2cb234c2abef0d7f57765eeffb67331814bdb01c590359325e)), latest: Some((11, 7054258f4310b7803419681db10cd4a47707c5d5182ec9317fae7e2e93a399af))
[2022-06-02T12:50:12Z DEBUG oura::filters::fingerprint] computed fingerprint 5.blck.170472701146953021650569694782289992283
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4245058320, payload length: 720
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_miniprotocols::chainsync::agents] rolling forward
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::run] rolling forward to point (12, 8b72b27a29fbe95f925f97afa0ce81c2e9aface13a539c733733153dadb8c928)
[2022-06-02T12:50:12Z DEBUG oura::sources::n2c::run] found 1 points with required min depth
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::run] rollback buffer state, size: 6, oldest: Some((7, 2d9136c363c69ad07e1a918de2ff5aeeba4361e33b9c2597511874f211ca26e9)), latest: Some((12, 8b72b27a29fbe95f925f97afa0ce81c2e9aface13a539c733733153dadb8c928))
[2022-06-02T12:50:12Z DEBUG oura::filters::fingerprint] computed fingerprint 6.blck.324557618939437755438504268445377141180
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4245068381, payload length: 720
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_miniprotocols::chainsync::agents] rolling forward
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::run] rolling forward to point (13, 5701cdc2396dc3f3ee32a13ea538d9a24059c3d00e7b928da3172835504001a9)
[2022-06-02T12:50:12Z DEBUG oura::sources::n2c::run] found 1 points with required min depth
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::run] rollback buffer state, size: 6, oldest: Some((8, f4b2213e2ed6dc8d4da0f4234ced91714c8cbaa0f3750abb90d10aff23ce4d67)), latest: Some((13, 5701cdc2396dc3f3ee32a13ea538d9a24059c3d00e7b928da3172835504001a9))
[2022-06-02T12:50:12Z DEBUG oura::filters::fingerprint] computed fingerprint 7.blck.126028998612995623777087803831947028296
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4245078470, payload length: 720
[2022-06-02T12:50:12Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:12Z DEBUG pallas_miniprotocols::chainsync::agents] rolling forward
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::run] rolling forward to point (14, 8ee800d2d2cb242a5fe96a13371118a3ceb1cd068d5daa91385c351954bece35)
[2022-06-02T12:50:12Z DEBUG oura::sources::n2c::run] found 1 points with required min depth
[2022-06-02T12:50:12Z INFO  oura::sources::n2c::run] rollback buffer state, size: 6, oldest: Some((9, 9cf9488161d63c7c414008737e1fab2eb8b85bf55dfe3fd8a09d2bc0c4327fe3)), latest: Some((14, 8ee800d2d2cb242a5fe96a13371118a3ceb1cd068d5daa91385c351954bece35))
[2022-06-02T12:50:12Z DEBUG oura::filters::fingerprint] computed fingerprint 8.blck.315131429030817542879403118618822272032
[2022-06-02T12:50:13Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4245088545, payload length: 720
[2022-06-02T12:50:13Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:13Z DEBUG pallas_miniprotocols::chainsync::agents] rolling forward
[2022-06-02T12:50:13Z INFO  oura::sources::n2c::run] rolling forward to point (15, d7900f547f96465c55ea3d8e6a9f6aeb88be200ad2138540eee716e94c1d211c)
[2022-06-02T12:50:13Z DEBUG oura::sources::n2c::run] found 1 points with required min depth
[2022-06-02T12:50:13Z INFO  oura::sources::n2c::run] rollback buffer state, size: 6, oldest: Some((10, 4bcd56b2e530a9690a94fb12a03cfd2cb7f25cdb229fda545c8cfddb57e13e24)), latest: Some((15, d7900f547f96465c55ea3d8e6a9f6aeb88be200ad2138540eee716e94c1d211c))
[2022-06-02T12:50:13Z DEBUG oura::filters::fingerprint] computed fingerprint 9.blck.28307267586314634055945253171528172206
[2022-06-02T12:50:13Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4245098662, payload length: 720
[2022-06-02T12:50:13Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:13Z DEBUG pallas_miniprotocols::chainsync::agents] rolling forward
[2022-06-02T12:50:13Z INFO  oura::sources::n2c::run] rolling forward to point (16, d0d37e4c9eabfe893c9c2cdaab7e90d450e570462a4dd315b6e7a6dcb54bd29a)
[2022-06-02T12:50:13Z DEBUG oura::sources::n2c::run] found 1 points with required min depth
[2022-06-02T12:50:13Z INFO  oura::sources::n2c::run] rollback buffer state, size: 6, oldest: Some((11, 7054258f4310b7803419681db10cd4a47707c5d5182ec9317fae7e2e93a399af)), latest: Some((16, d0d37e4c9eabfe893c9c2cdaab7e90d450e570462a4dd315b6e7a6dcb54bd29a))
[2022-06-02T12:50:13Z DEBUG oura::filters::fingerprint] computed fingerprint 10.blck.110053003063584745778902325309990952554
[2022-06-02T12:50:13Z DEBUG pallas_multiplexer::bearers] parsed inbound msg, protocol id: 5, ts: 4245108749, payload length: 720
[2022-06-02T12:50:13Z DEBUG pallas_multiplexer] successful tx to egress protocol
[2022-06-02T12:50:13Z DEBUG pallas_miniprotocols::chainsync::agents] rolling forward
[2022-06-02T12:50:13Z INFO  oura::sources::n2c::run] rolling forward to point (17, 0be303663930359a7c2d2d039458cd537110863c24f475bffb09b19dc3f9d54a)
[2022-06-02T12:50:13Z DEBUG oura::sources::n2c::run] found 1 points with required min depth
[2022-06-02T12:50:13Z INFO  oura::sources::n2c::run] rollback buffer state, size: 6, oldest: Some((12, 8b72b27a29fbe95f925f97afa0ce81c2e9aface13a539c733733153dadb8c928)), latest: Some((17, 0be303663930359a7c2d2d039458cd537110863c24f475bffb09b19dc3f9d54a))
[2022-06-02T12:50:13Z DEBUG oura::filters::fingerprint] computed fingerprint 11.blck.42614170289786446319973403040355468820
ERROR: Error(
    Io(
        Error {
            kind: UnexpectedEof,
            message: "failed to fill whole buffer",
        },
    ),
    State {
        next_error: None,
        backtrace: None,
    },
)
scarmuega commented 2 years ago

Thanks for reporting @PegasusPool. We'll try to reproduce with this exact config and get back to you.

Can you confirm that your Kafka broker is reachable at the configure location? Are you running from inside a docker container?

One possibility is that the unexpected EOF could be an issue reading from the TCP socket that connects to the broker.

From the logs I can confirm that the pipeline segment in charge of reading from Cardano node is working as expected.

PegasusPool commented 2 years ago

Thanks for the quick response @scarmuega . I checked the connection using kafkacat and I got this:

kafkacat -b xxx:9092 -t cardano-events
% Auto-selecting Consumer mode (use -P or -C to override)
% ERROR: Failed to query metadata for topic cardano-events: Local: Broker transport failure

It seems like the issue is on the kafka config side. I'm quite new to Kafka to be honest, not sure what I missed when I setup the cluster and the topic.

I'm not running from inside a docker container.

I wanted to post this also in case this helps:

RUST_BACKTRACE=1 oura daemon
ERROR: Error(
    Io(
        Error {
            kind: UnexpectedEof,
            message: "failed to fill whole buffer",
        },
    ),
    State {
        next_error: None,
        backtrace: Some(
               0:     0x564f62a22517 - <unknown>
               1:     0x564f62a11865 - backtrace::capture::Backtrace::new::h09c254d9d81176ce
               2:     0x564f62a111cc - error_chain::make_backtrace::h93194344469217d7
               3:     0x564f62a11286 - <error_chain::State as core::default::Default>::default::h1f860986a1067232
               4:     0x564f62a0677d - <unknown>
               5:     0x564f629fb104 - kafka::client::__get_response_size::h179a659b657009fd
               6:     0x564f622626ab - <unknown>
               7:     0x564f6236c947 - <unknown>
               8:     0x564f622aebf1 - oura::sinks::kafka::setup::<impl oura::pipelining::SinkProvider for oura::utils::WithUtils<oura::sinks::kafka::setup::Config>>::bootstrap::h0fe2c846ccb4ad1a
               9:     0x564f62229d18 - <unknown>
              10:     0x564f62246b2c - <unknown>
              11:     0x564f622398a3 - <unknown>
              12:     0x564f6223d629 - <unknown>
              13:     0x564f62f1f5b1 - std::rt::lang_start_internal::h2ba92edce36c035e
              14:     0x564f622488d2 - <unknown>
              15:     0x7ff8c0ca20b3 - __libc_start_main
              16:     0x564f621d6439 - <unknown>
              17:                0x0 - <unknown>
            ,
        ),
    },
)
scarmuega commented 2 years ago

I would revisit the topic creation procedure. It usually goes something like this:

/bin/kafka-topics.sh --create \
    --zookeeper <hostname>:<port> \
    --topic <topic-name> \
    --partitions <number-of-partitions> \
    --replication-factor <number-of-replicating-servers>

I also recommend checking out https://redpanda.com/, it's a one-binary, self-contained, zookeeper-free tool that is kafka-compatible. From my experience, it's easier to setup and manage.

PegasusPool commented 2 years ago

Thanks for the suggestions @scarmuega

I downloaded a sample code for a client and I configured it to produce messages for my cluster and topic and it worked. The messages arrived fine in the cluster which is suggesting me that Oura should also be able to send messages.

Does this bit look all right to you?

[sink]
type = "Kafka"
brokers = ["pkc-3w22w.us-central1.gcp.confluent.cloud:9092"]
topic = "cardano-events"

One thing I had to configure in the sample app to produce messages is the api key and secret of the cluster which I'm not sure why I'm not required to provide when configuring Oura. Here is the config for the sample app that's working:

# Required connection configs for Kafka producer, consumer, and admin
bootstrap.servers=pkc-3w22w.us-central1.gcp.confluent.cloud:9092
security.protocol=SASL_SSL
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username='API_KEY_HERE' password='API_SECRET_HERE';
sasl.mechanism=PLAIN
# Required for correctness in Apache Kafka clients prior to 2.6
client.dns.lookup=use_all_dns_ips

# Best practice for higher availability in Apache Kafka clients prior to 3.0
session.timeout.ms=45000
PegasusPool commented 2 years ago

Interestingly if I create a local kafka server Oura is able to send messages to that just fine. :thinking:

PegasusPool commented 2 years ago

I ended up just using a local kafka server, so I'll close this issue for now. Thanks for looking into this anyway