search

privacylab / talek

a Private Publish Subscribe System
BSD 2-Clause "Simplified" License
47 stars 5 forks source link

bug: talekreplica replica.go:110: Failed to decode part of batch read EOF [at index 0] #105

Closed mixmasala closed 8 months ago

mixmasala commented 8 months ago 
e6d0b67b79b7:/build/talek# podman logs talek_net-frontend-1 
2024-03-11 12:05:47.476041 I | ----------------------
2024-03-11 12:05:47.476154 I | --- Talek Frontend ---
2024-03-11 12:05:47.476156 I | ----------------------
2024-03-11 12:05:47.476585 I | Running.
[Frontend:Talek Frontend] 2024/03/11 12:05:48 frontend.go:144: Periodic update of database sent to replicas.
[Frontend:Talek Frontend] 2024/03/11 12:05:49 frontend.go:144: Periodic update of database sent to replicas.
[Frontend:Talek Frontend] 2024/03/11 12:05:50 frontend.go:144: Periodic update of database sent to replicas.
[Frontend:Talek Frontend] 2024/03/11 12:05:51 frontend.go:144: Periodic update of database sent to replicas.
[Frontend:Talek Frontend] 2024/03/11 12:05:52 frontend.go:144: Periodic update of database sent to replicas.
[Frontend:Talek Frontend] 2024/03/11 12:05:53 frontend.go:144: Periodic update of database sent to replicas.
[Frontend:Talek Frontend] 2024/03/11 12:05:54 frontend.go:144: Periodic update of database sent to replicas.
[Frontend:Talek Frontend] 2024/03/11 12:05:55 frontend.go:144: Periodic update of database sent to replicas.
[Frontend:Talek Frontend] 2024/03/11 12:05:56 frontend.go:144: Periodic update of database sent to replicas.
[Frontend:Talek Frontend] 2024/03/11 12:05:57 frontend.go:144: Periodic update of database sent to replicas.
[Frontend:Talek Frontend] 2024/03/11 12:05:58 frontend.go:144: Periodic update of database sent to replicas.
[Frontend:Talek Frontend] 2024/03/11 12:05:59 frontend.go:144: Periodic update of database sent to replicas.
[Frontend:Talek Frontend] 2024/03/11 12:06:00 frontend.go:144: Periodic update of database sent to replicas.
[Frontend:Talek Frontend] 2024/03/11 12:06:01 frontend.go:99: write to 78,330 serialized.
[Frontend:Talek Frontend] 2024/03/11 12:06:01 frontend.go:226: Read: add to batch, size=1
[Frontend:Talek Frontend] 2024/03/11 12:06:01 frontend.go:250: Batch read with 1 items sent to replicas.
[Frontend:Talek Frontend] 2024/03/11 12:06:01 frontend.go:271: Error making read to replica 0: Post "http://localhost:8081": EOF

and the replica exited with the error "Failed to decode part of batch read EOF"

e6d0b67b79b7:/build/talek# podman logs talek_net-replica1-1 
2024-03-11 12:05:47.437514 I | ---------------------
2024-03-11 12:05:47.437632 I | --- Talek Replica ---
2024-03-11 12:05:47.437634 I | ---------------------
2024-03-11 12:05:47.437647 I | Arguments:
2024-03-11 12:05:47.437650 I | config=/talek_shared/replica1.json
2024-03-11 12:05:47.437652 I | backing=cpu.0
2024-03-11 12:05:47.437857 I | Using the following configuration:
2024-03-11 12:05:47.437870 I | serverConfig=server.Config{Config:(*common.Config)(0xc00016c000), ReadBatch:8, WriteInterval:1000000000, ReadInterval:1000000000, TrustDomain:(*common.TrustDomainConfig)(0xc0001680d0), TrustDomainIndex:1}
2024-03-11 12:05:47.437883 I | serverConfig.Config=&common.Config{NumBuckets:0x400, BucketDepth:0x4, DataSize:0x400, BloomFalsePositive:0.05, WriteInterval:1000000000, ReadInterval:1000000000, InterestMultiple:0xa, InterestSeed:-5534542852170495829, MaxLoadFactor:0.95, LoadFactorStep:0.05}
[CPU Shard (cpu.0)] INFO: 2024/03/11 12:05:47 shard_cpu.go:76: NewShardCPU(CPU Shard (cpu.0)) finished
2024-03-11 12:05:47.440606 I | Running.
[replica1] TRACE: 2024/03/11 12:05:48 replica.go:67: Write: enter
[replica1] TRACE: 2024/03/11 12:05:48 shard.go:101: Write: 
[replica1] TRACE: 2024/03/11 12:05:48 replica.go:84: Write: exit
[CPU Shard (cpu.0)] INFO: 2024/03/11 12:05:48 shard_cpu.go:82: CPU Shard (cpu.0).Free finished
[CPU Shard (cpu.0)] INFO: 2024/03/11 12:05:48 shard_cpu.go:76: NewShardCPU(CPU Shard (cpu.0)) finished
[replica1] TRACE: 2024/03/11 12:05:49 replica.go:67: Write: enter
[replica1] TRACE: 2024/03/11 12:05:49 shard.go:101: Write: 
[replica1] TRACE: 2024/03/11 12:05:49 replica.go:84: Write: exit
[CPU Shard (cpu.0)] INFO: 2024/03/11 12:05:49 shard_cpu.go:82: CPU Shard (cpu.0).Free finished
[CPU Shard (cpu.0)] INFO: 2024/03/11 12:05:49 shard_cpu.go:76: NewShardCPU(CPU Shard (cpu.0)) finished
[replica1] TRACE: 2024/03/11 12:05:50 replica.go:67: Write: enter
[replica1] TRACE: 2024/03/11 12:05:50 shard.go:101: Write: 
[replica1] TRACE: 2024/03/11 12:05:50 replica.go:84: Write: exit
[CPU Shard (cpu.0)] INFO: 2024/03/11 12:05:50 shard_cpu.go:82: CPU Shard (cpu.0).Free finished
[CPU Shard (cpu.0)] INFO: 2024/03/11 12:05:50 shard_cpu.go:76: NewShardCPU(CPU Shard (cpu.0)) finished
[replica1] TRACE: 2024/03/11 12:05:51 replica.go:67: Write: enter
[replica1] TRACE: 2024/03/11 12:05:51 shard.go:101: Write: 
[replica1] TRACE: 2024/03/11 12:05:51 replica.go:84: Write: exit
[CPU Shard (cpu.0)] INFO: 2024/03/11 12:05:51 shard_cpu.go:82: CPU Shard (cpu.0).Free finished
[CPU Shard (cpu.0)] INFO: 2024/03/11 12:05:51 shard_cpu.go:76: NewShardCPU(CPU Shard (cpu.0)) finished
[replica1] TRACE: 2024/03/11 12:05:52 replica.go:67: Write: enter
[replica1] TRACE: 2024/03/11 12:05:52 shard.go:101: Write: 
[replica1] TRACE: 2024/03/11 12:05:52 replica.go:84: Write: exit
[CPU Shard (cpu.0)] INFO: 2024/03/11 12:05:52 shard_cpu.go:82: CPU Shard (cpu.0).Free finished
[CPU Shard (cpu.0)] INFO: 2024/03/11 12:05:52 shard_cpu.go:76: NewShardCPU(CPU Shard (cpu.0)) finished
[replica1] TRACE: 2024/03/11 12:05:53 replica.go:67: Write: enter
[replica1] TRACE: 2024/03/11 12:05:53 shard.go:101: Write: 
[replica1] TRACE: 2024/03/11 12:05:53 replica.go:84: Write: exit
[CPU Shard (cpu.0)] INFO: 2024/03/11 12:05:53 shard_cpu.go:82: CPU Shard (cpu.0).Free finished
[CPU Shard (cpu.0)] INFO: 2024/03/11 12:05:53 shard_cpu.go:76: NewShardCPU(CPU Shard (cpu.0)) finished
[replica1] TRACE: 2024/03/11 12:05:54 replica.go:67: Write: enter
[replica1] TRACE: 2024/03/11 12:05:54 shard.go:101: Write: 
[replica1] TRACE: 2024/03/11 12:05:54 replica.go:84: Write: exit
[CPU Shard (cpu.0)] INFO: 2024/03/11 12:05:54 shard_cpu.go:82: CPU Shard (cpu.0).Free finished
[CPU Shard (cpu.0)] INFO: 2024/03/11 12:05:54 shard_cpu.go:76: NewShardCPU(CPU Shard (cpu.0)) finished
[replica1] TRACE: 2024/03/11 12:05:55 replica.go:67: Write: enter
[replica1] TRACE: 2024/03/11 12:05:55 shard.go:101: Write: 
[replica1] TRACE: 2024/03/11 12:05:55 replica.go:84: Write: exit
[CPU Shard (cpu.0)] INFO: 2024/03/11 12:05:55 shard_cpu.go:82: CPU Shard (cpu.0).Free finished
[CPU Shard (cpu.0)] INFO: 2024/03/11 12:05:55 shard_cpu.go:76: NewShardCPU(CPU Shard (cpu.0)) finished
[replica1] TRACE: 2024/03/11 12:05:56 replica.go:67: Write: enter
[replica1] TRACE: 2024/03/11 12:05:56 shard.go:101: Write: 
[replica1] TRACE: 2024/03/11 12:05:56 replica.go:84: Write: exit
[CPU Shard (cpu.0)] INFO: 2024/03/11 12:05:56 shard_cpu.go:82: CPU Shard (cpu.0).Free finished
[CPU Shard (cpu.0)] INFO: 2024/03/11 12:05:56 shard_cpu.go:76: NewShardCPU(CPU Shard (cpu.0)) finished
[replica1] TRACE: 2024/03/11 12:05:57 replica.go:67: Write: enter
[replica1] TRACE: 2024/03/11 12:05:57 shard.go:101: Write: 
[replica1] TRACE: 2024/03/11 12:05:57 replica.go:84: Write: exit
[CPU Shard (cpu.0)] INFO: 2024/03/11 12:05:57 shard_cpu.go:82: CPU Shard (cpu.0).Free finished
[CPU Shard (cpu.0)] INFO: 2024/03/11 12:05:57 shard_cpu.go:76: NewShardCPU(CPU Shard (cpu.0)) finished
[replica1] TRACE: 2024/03/11 12:05:58 replica.go:67: Write: enter
[replica1] TRACE: 2024/03/11 12:05:58 shard.go:101: Write: 
[replica1] TRACE: 2024/03/11 12:05:58 replica.go:84: Write: exit
[CPU Shard (cpu.0)] INFO: 2024/03/11 12:05:58 shard_cpu.go:82: CPU Shard (cpu.0).Free finished
[CPU Shard (cpu.0)] INFO: 2024/03/11 12:05:58 shard_cpu.go:76: NewShardCPU(CPU Shard (cpu.0)) finished
[replica1] TRACE: 2024/03/11 12:05:59 replica.go:67: Write: enter
[replica1] TRACE: 2024/03/11 12:05:59 shard.go:101: Write: 
[replica1] TRACE: 2024/03/11 12:05:59 replica.go:84: Write: exit
[CPU Shard (cpu.0)] INFO: 2024/03/11 12:05:59 shard_cpu.go:82: CPU Shard (cpu.0).Free finished
[CPU Shard (cpu.0)] INFO: 2024/03/11 12:05:59 shard_cpu.go:76: NewShardCPU(CPU Shard (cpu.0)) finished
[replica1] TRACE: 2024/03/11 12:06:00 replica.go:67: Write: enter
[replica1] TRACE: 2024/03/11 12:06:00 shard.go:101: Write: 
[replica1] TRACE: 2024/03/11 12:06:00 replica.go:84: Write: exit
[CPU Shard (cpu.0)] INFO: 2024/03/11 12:06:00 shard_cpu.go:82: CPU Shard (cpu.0).Free finished
[CPU Shard (cpu.0)] INFO: 2024/03/11 12:06:00 shard_cpu.go:76: NewShardCPU(CPU Shard (cpu.0)) finished
[replica1] TRACE: 2024/03/11 12:06:01 replica.go:67: Write: enter
[replica1] TRACE: 2024/03/11 12:06:01 shard.go:101: Write: 
[replica1] TRACE: 2024/03/11 12:06:01 replica.go:84: Write: exit
[replica1] TRACE: 2024/03/11 12:06:01 replica.go:91: BatchRead: enter
[replica1] ERROR: 2024/03/11 12:06:01 replica.go:110: Failed to decode part of batch read EOF [at index 0]
mixmasala commented 8 months ago

I think this was related to talekutil default values of flags overwriting the trustdomain index (--index) when creating replica public configuration