search

MinaProtocol / mina

Mina is a cryptocurrency protocol with a constant size blockchain, improving scaling while maintaining decentralization and security.
https://minaprotocol.com
Apache License 2.0
1.99k stars 529 forks source link

Coda sandbox crashed with (epoch, slot) did not increase #5924

Closed trader-payne closed 2 years ago

trader-payne commented 4 years ago

Attached the core dump. coda_crash_report_2020-09-12_09-30-13.245207.tar.gz

Logs:

2020-09-11 19:43:22 UTC [Info] would have validated but it's from us!
2020-09-11 19:44:27 UTC [Info] Coda process was interrupted by "sigterm"
2020-09-11 19:44:27 UTC [Info] Running async shutdown handler: "Close transition frontier, if exists"
2020-09-12 09:27:13 UTC [Info] Coda daemon is booting up; built with commit "02d0fd452ae55ee8a2501f8a4e0e703cc38e5bac" on branch "feature/release-fix-leaks"
2020-09-12 09:27:13 UTC [Info] Daemon will expire at "2024-12-10 14:00:00-07:00"
2020-09-12 09:27:13 UTC [Info] Booting may take several seconds, please wait
2020-09-12 09:27:13 UTC [Info] Initializing with runtime configuration $config
        config: {
  "ledger": {
    "accounts": [
      {
        "pk": "B62qrPN5Y5yq8kGE3FbVKbGTdTAJNdtNtB5sNVpxyRwWGcDEhpMzc8g",
        "balance": "66000"
      }
    ]
  }
}
2020-09-12 09:27:13 UTC [Info] Using the compiled constraint constants
2020-09-12 09:27:13 UTC [Info] Found ledger file at $path
        path: "/var/lib/coda/genesis_ledger_accounts_a5d8a4777218092b8046dc7178aa19c90b6626ba0b72d62d966a34b94423acbe_0bb3b0dc74e9d44390f973c04ef3ca4d32d8d51279bc5a8ddedc2cc5d7915d37.tar.gz"
2020-09-12 09:27:13 UTC [Info] Loading genesis ledger from $path
        path: "/var/lib/coda/genesis_ledger_accounts_a5d8a4777218092b8046dc7178aa19c90b6626ba0b72d62d966a34b94423acbe_0bb3b0dc74e9d44390f973c04ef3ca4d32d8d51279bc5a8ddedc2cc5d7915d37.tar.gz"
2020-09-12 09:27:13 UTC [Info] Genesis proof file $path does not exist
        path: "/root/.coda-config/genesis_proof_3b1751b8d3c4cebdbbb901e1cd69386941f9004447fa73bdb7c61b8296eaa71e"
2020-09-12 09:27:13 UTC [Info] Found genesis proof file at $path
        path: "/var/lib/coda/genesis_proof_3b1751b8d3c4cebdbbb901e1cd69386941f9004447fa73bdb7c61b8296eaa71e"
2020-09-12 09:27:13 UTC [Info] Loaded ledger from $ledger_file and genesis proof from $proof_file
        ledger_file: "/var/lib/coda/genesis_ledger_accounts_a5d8a4777218092b8046dc7178aa19c90b6626ba0b72d62d966a34b94423acbe_0bb3b0dc74e9d44390f973c04ef3ca4d32d8d51279bc5a8ddedc2cc5d7915d37.tar.gz"
        proof_file: "/var/lib/coda/genesis_proof_3b1751b8d3c4cebdbbb901e1cd69386941f9004447fa73bdb7c61b8296eaa71e"
2020-09-12 09:27:14 UTC [Info] Starting node as a seed node
2020-09-12 09:27:14 UTC [Info] Setting current protocol version to "0.1.0" from compile-time config
2020-09-12 09:28:11 UTC [Info] Daemon started process of kind "Prover" with pid 129
2020-09-12 09:28:12 UTC [Info] Daemon started process of kind "Verifier" with pid 182
2020-09-12 09:28:12 UTC [Info] Registering async shutdown handler: "Close transition frontier, if exists"
2020-09-12 09:28:12 UTC [Info] libp2p peer ID this session is $peer_id
        peer_id: "12D3KooWH6sEB6mrndQbuXXUBQFedHcRjKwvZVGfT834zkG2RgpP"
2020-09-12 09:28:12 UTC [Info] All 0 tables opened in 0s

2020-09-12 09:28:12 UTC [Info] Replaying file id: 0 at offset: 0

2020-09-12 09:28:12 UTC [Info] Replay took: 133.401µs

2020-09-12 09:28:12 UTC [Info] All 0 tables opened in 0s

2020-09-12 09:28:12 UTC [Info] Replaying file id: 0 at offset: 0

2020-09-12 09:28:12 UTC [Info] Replay took: 6.57µs

2020-09-12 09:28:12 UTC [Info] starting refreshing cpl 0 with key CIQAAAFIO4AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA (routing table size was 0)
2020-09-12 09:28:12 UTC [Warn] failed when refreshing routing table2 errors occurred:
        * failed to query for self, err=failed to find any peer in table
        * failed to refresh cpl=0, err=failed to find any peer in table

2020-09-12 09:28:12 UTC [Info] starting refreshing cpl 0 with key CIQAAACAR4AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA (routing table size was 0)
2020-09-12 09:28:12 UTC [Warn] failed when refreshing routing table2 errors occurred:
        * failed to query for self, err=failed to find any peer in table
        * failed to refresh cpl=0, err=failed to find any peer in table

2020-09-12 09:28:12 UTC [Info] here are the seeds: []
2020-09-12 09:28:12 UTC [Info] starting refreshing cpl 0 with key CIQAAAKFR4AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA (routing table size was 0)
2020-09-12 09:28:12 UTC [Info] failed when refreshing routing table
2020-09-12 09:28:12 UTC [Info] starting refreshing cpl 0 with key CIQAAAB7TEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA (routing table size was 0)
2020-09-12 09:28:12 UTC [Info] failed when refreshing routing table
2020-09-12 09:28:12 UTC [Info] hacking peers: "()"
2020-09-12 09:28:12 UTC [Warn] Persistent frontier hash did not match persistent root frontier hash (resetting frontier hash)
2020-09-12 09:28:12 UTC [Info] starting refreshing cpl 0 with key CIQAAAGWEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA (routing table size was 0)
2020-09-12 09:28:12 UTC [Info] starting refreshing cpl 0 with key CIQAAAAUWAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA (routing table size was 0)
2020-09-12 09:28:12 UTC [Info] failed when refreshing routing table
2020-09-12 09:28:12 UTC [Info] failed when refreshing routing table
2020-09-12 09:28:12 UTC [Info] Pausing block production while bootstrapping
2020-09-12 09:28:12 UTC [Info] Created GraphQL server at: http://localhost:3085/graphql
2020-09-12 09:28:12 UTC [Info] Requesting peers for their best tip to do initialization
2020-09-12 09:28:12 UTC [Info] Initializing plugins
2020-09-12 09:28:12 UTC [Info] Daemon ready. Clients can now connect
2020-09-12 09:28:17 UTC [Info] DiscoverNAT error:no NAT found
2020-09-12 09:28:18 UTC [Info] Starting Transition Frontier Controller phase
2020-09-12 09:28:18 UTC [Info] Updating new available work took 4.675865173339844 ms
2020-09-12 09:28:18 UTC [Info] Determining next slot to produce block
2020-09-12 09:28:18 UTC [Info] Checking VRF evaluations at epoch: 0, slot: 1
2020-09-12 09:28:18 UTC [Info] Producing block in 1 slots
Transaction_snark.system: 19.28s
{"timestamp":"2020-09-12 09:28:32.120693Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File \"src/lib/snark_worker/functor.ml\", line 176, characters 10-21"},"message":"SNARK work $work_ids received from $address. Starting proof generation","metadata":{"address":"127.0.0.1:8301","pid":242,"process":"Snark Worker","work_ids":[379804077,936229598]}}
2020-09-12 09:30:13 UTC [Fatal] Unhandled top-level exception: $exn
Generating crash report
        exn: "(monitor.ml.Error \"(epoch, slot) did not increase. prev=(0 70), next=(0 1)\"\n  (\"Raised at file \\\"src/error.ml\\\" (inlined), line 9, characters 14-30\"\n    \"Called from file \\\"src/or_error.ml\\\", line 72, characters 17-32\"\n    \"Called from file \\\"src/lib/consensus/proof_of_stake.ml\\\", line 3088, characters 8-291\"\n    \"Called from file \\\"src/lib/block_producer/block_producer.ml\\\", line 62, characters 49-55\"\n    \"Called from file \\\"src/deferred0.ml\\\", line 49, characters 2-10\"\n    \"Called from file \\\"src/lib/block_producer/block_producer.ml\\\", line 62, characters 4-57\"\n    \"Called from file \\\"src/lib/block_producer/block_producer.ml\\\", line 183, characters 8-1023\"\n    \"Called from file \\\"src/lib/interruptible/interruptible.ml\\\", line 24, characters 23-26\"\n    \"Called from file \\\"src/result.ml\\\", line 96, characters 19-24\"\n    \"Called from file \\\"src/deferred1.ml\\\", line 17, characters 40-45\"\n    \"Called from file \\\"src/job_queue.ml\\\" (inlined), line 131, characters 2-5\"\n    \"Called from file \\\"src/job_queue.ml\\\", line 171, characters 6-47\"\n    \"Caught by monitor coda\"))"

  ☠  Coda Daemon crashed.
   The Coda Protocol developers would like to know why!

    Please:
      Open an issue:
        <https://github.com/CodaProtocol/coda/issues/new>

      Briefly describe what you were doing and attach the crash report /root/.coda-config/coda_crash_report_2020-09-12_09-30-13.245207.tar.gz
garethtdavies commented 4 years ago

This error occurs when restarting a stopped container as the offset appears to not being set correctly and hence is using the hardcoded genesis time. https://github.com/CodaProtocol/coda/blob/develop/scripts/run-demo.sh

The solution is simply to remove the sandbox container docker rm -f coda and start a new one.

shimkiv commented 2 years ago

Closing this issue as it is stale for a long time.