search

HelixNetwork / pendulum

Pendulum is a distributed messaging protocol that enables globally available tamper proof timestamps :hourglass_flowing_sand:
https://dev.hlx.ai
Other
10 stars 6 forks source link

Current network issue - validation fails for validator milestone #160

Closed dnck closed 4 years ago

dnck commented 4 years ago

Within the first 20 minutes of running, the log output of nominee1.hlxtest.net starts to either receive/produce invalid milestones.

According to the log, this is a result of a balance being inconsistent:

09/22 18:04:13.162 [Latest Milestone Tracker] DEBUG MilestoneTrackerImpl:391 - Process Milestone 007f6591b19103acdc62b7b49e42097f82d7e13688b61def0c0e594bfb31a82a, round: 29965

09/22 18:04:13.163 [Latest Milestone Tracker] DEBUG MilestoneTrackerImpl:170 - New milestone 007f6591b19103acdc62b7b49e42097f82d7e13688b61def0c0e594bfb31a82a (1/6) added to round #29965

09/22 18:04:13.437 [pool-20-thread-1] DEBUG MilestonePublisher:186 - Publishing next Milestone...

09/22 18:04:13.440 [pool-20-thread-1] DEBUG WalkValidator:85 - Validation failed: 007f6591b19103acdc62b7b49e42097f82d7e13688b61def0c0e594bfb31a82a balance is not consistent

...

09/22 18:04:28.200 [Latest Milestone Tracker] DEBUG MilestoneTrackerImpl:391 - Process Milestone 0039e989c89c05eae82b4fd4667e344711a37e1e843e44fdbe5e5ec39adc3fa9, round: 29966

09/22 18:04:28.201 [Latest Milestone Tracker] DEBUG MilestoneTrackerImpl:170 - New milestone 0039e989c89c05eae82b4fd4667e344711a37e1e843e44fdbe5e5ec39adc3fa9 (1/6) added to round #29966

09/22 18:04:28.460 [pool-20-thread-1] DEBUG MilestonePublisher:186 - Publishing next Milestone...

09/22 18:04:28.465 [pool-20-thread-1] DEBUG WalkValidator:85 - Validation failed: 0039e989c89c05eae82b4fd4667e344711a37e1e843e44fdbe5e5ec39adc3fa9 balance is not consistent
oracle58 commented 4 years ago

Would be better to try and reproduce it locally, imo.

oracle58 commented 4 years ago

related to #142

cristina-vasiu commented 4 years ago

@dnck is this issue stil valid on current mainnet? Are there any stepts how to reproduced it locally (how many nodes, how many nominees)?

dnck commented 4 years ago

@dnck is this issue stil valid on current mainnet? Are there any stepts how to reproduced it locally (how many nodes, how many nominees)?

I have a docker set up

https://github.com/dnck/helix-docker-cluster

It runs the five nominees, and it breaks on all value transactions.