fix: bug on lagrange coeff computation

alxiong commented 3 months ago

Closes #1632

This PR:

Fix the edge case when zeta is one of the evaluation domain elements, and our lagrange coefficient (and also pi_poly_eval) is computed incorrectly. Tests are also added for these rare edge cases.

alxiong commented 3 months ago

I find the lint errors unexpected. it has nothing to do with my change, and when I run in my nix shell, the same command doesn't produce any errors.

is this because I'm branching out from a historical commit and these CI failures are fixed later on main?

(also I'll wait for at least another reviewer from the contract team, preferably @philippecamacho? before merging)

philippecamacho commented 3 months ago

Here is the error I have locally:

> just test

running 3 tests
test test::test_non_voting_hotshot_node ... ignored
test non_permissioned::test::test_non_permissioned_builder ... FAILED
test permissioned::test::test_permissioned_builder ... ok

failures:

---- non_permissioned::test::test_non_permissioned_builder stdout ----
  2024-06-29T00:29:21.709739Z  INFO hotshot::traits::networking::memory_network: Attaching new MemoryNetwork
    at /home/leloup/.cargo-nix/git/checkouts/hotshot-0ac703037ea89b98/f02711d/crates/hotshot/src/traits/networking/memory_network.rs:116
    in hotshot::traits::networking::memory_network::new with pub_key: VerKey((QuadExtField(19567118040283358755881717554180281510990490882189116154441543720028451736056 + 14769573949975580113492449607617874378387493209520719723082704337724941814257 * u), QuadExtField(6693550608351406366602178065881122423608118597467142574782141406801466584576 + 20531268181388740562787463379713125736669669264177461912768443159187228286464 * u), QuadExtField(2355642833004872935157132422655966544275915092657230564756215612621792009326 + 17365009061808721819595011285181318602703685327462243792288958758516746215676 * u))), master_map: MasterMap { id: 2930681509702876756 }, reliability_config: None

  2024-06-29T00:29:21.709958Z  INFO builder::testing: Before init hotshot
    at builder/src/lib.rs:402

  2024-06-29T00:29:21.930430Z  INFO builder::testing: After init hotshot
    at builder/src/lib.rs:417

  2024-06-29T00:29:21.930461Z  INFO hotshot::traits::networking::memory_network: Attaching new MemoryNetwork
    at /home/leloup/.cargo-nix/git/checkouts/hotshot-0ac703037ea89b98/f02711d/crates/hotshot/src/traits/networking/memory_network.rs:116
    in hotshot::traits::networking::memory_network::new with pub_key: VerKey((QuadExtField(20206540741426680705134601859944499051253071221295568556091720238236899384780 + 21214762758829364234963164365500570367893124135218375411573030470750627100993 * u), QuadExtField(4187678195349084957605093805698157923299421136390768416368356816190265868598 + 10080927991155731570453897812984439100910909897510390255046233387537879108727 * u), QuadExtField(15594834959918623473952205907777587894941730045666030070321543329863088070288 + 11382585529949620306749703123125219600724963431548092718219601218482742929123 * u))), master_map: MasterMap { id: 2930681509702876756 }, reliability_config: None

  2024-06-29T00:29:21.930646Z  INFO builder::testing: Before init hotshot
    at builder/src/lib.rs:402

  2024-06-29T00:29:21.969409Z  INFO builder::testing: After init hotshot
    at builder/src/lib.rs:417

  2024-06-29T00:29:21.969431Z  INFO hotshot::traits::networking::memory_network: Attaching new MemoryNetwork
    at /home/leloup/.cargo-nix/git/checkouts/hotshot-0ac703037ea89b98/f02711d/crates/hotshot/src/traits/networking/memory_network.rs:116
    in hotshot::traits::networking::memory_network::new with pub_key: VerKey((QuadExtField(10273395821759281733467861264364014100521108975511354552733902621671820957008 + 11516070564807907130738292074360220707732285717128432780720084698204934159778 * u), QuadExtField(1097212724123606650008014127714195107927524210006117280136650721263274208397 + 18753302559051123211175456858988010090181832317953684257175355451104988110518 * u), QuadExtField(19989685632262209663983223599260095742950832913348586488504808041572839712544 + 11358275030085530654800672364595622705112441088080850151638552547537118997220 * u))), master_map: MasterMap { id: 2930681509702876756 }, reliability_config: None

  2024-06-29T00:29:21.969574Z  INFO builder::testing: Before init hotshot
    at builder/src/lib.rs:402

  2024-06-29T00:29:22.006939Z  INFO builder::testing: After init hotshot
    at builder/src/lib.rs:417

  2024-06-29T00:29:22.006966Z  INFO hotshot::traits::networking::memory_network: Attaching new MemoryNetwork
    at /home/leloup/.cargo-nix/git/checkouts/hotshot-0ac703037ea89b98/f02711d/crates/hotshot/src/traits/networking/memory_network.rs:116
    in hotshot::traits::networking::memory_network::new with pub_key: VerKey((QuadExtField(15912255821796679952448139285096570167957333322460906653973132360777831075664 + 4027206456611929488265604414936646315828289501853376314577843564495389706761 * u), QuadExtField(20567533835033763235494154357066149055267730644535090739800401392566830989327 + 3187826315427891300809399682950487870257994124051301280058409233593553116753 * u), QuadExtField(971738089468379185942023583652533820369042491684914111311586006736059590534 + 4964834600094319481149436221099455567880735613627014807827223030849469387399 * u))), master_map: MasterMap { id: 2930681509702876756 }, reliability_config: None

  2024-06-29T00:29:22.007128Z  INFO builder::testing: Before init hotshot
    at builder/src/lib.rs:402

  2024-06-29T00:29:22.044919Z  INFO builder::testing: After init hotshot
    at builder/src/lib.rs:417

  2024-06-29T00:29:22.044941Z  INFO hotshot::traits::networking::memory_network: Attaching new MemoryNetwork
    at /home/leloup/.cargo-nix/git/checkouts/hotshot-0ac703037ea89b98/f02711d/crates/hotshot/src/traits/networking/memory_network.rs:116
    in hotshot::traits::networking::memory_network::new with pub_key: VerKey((QuadExtField(19193222483235554574375331017003763994353581066404081426919302576920817477794 + 3251766414091361618675624993336703396228051268714604846102914194579315856916 * u), QuadExtField(18283172836003152989789683304462390059755376699212794386670528680229270609360 + 15378702307017372156564135124947728168276867210069974825133190548249549560183 * u), QuadExtField(19641829759175131855421958461437185318723783361936481025561161186133227839869 + 9050204667314691839747618710931579809239723962172532775955483583035000249282 * u))), master_map: MasterMap { id: 2930681509702876756 }, reliability_config: None

  2024-06-29T00:29:22.045081Z  INFO builder::testing: Before init hotshot
    at builder/src/lib.rs:402

  2024-06-29T00:29:22.080880Z  INFO builder::testing: After init hotshot
    at builder/src/lib.rs:417

  2024-06-29T00:29:22.080901Z  INFO hotshot::traits::networking::memory_network: Attaching new MemoryNetwork
    at /home/leloup/.cargo-nix/git/checkouts/hotshot-0ac703037ea89b98/f02711d/crates/hotshot/src/traits/networking/memory_network.rs:116
    in hotshot::traits::networking::memory_network::new with pub_key: VerKey((QuadExtField(4362928359505807910667185088147983337083948155178648300502692999807543906842 + 1717451536894860019503720782630561739386903118887148433226299979925022346616 * u), QuadExtField(15746840372244133575970480628702743069679021261281064236362250617510728588586 + 9564147398053140632305346244995950497888812986038995550081985293649530650658 * u), QuadExtField(14678494121518848842738999202252260111882977910423877718587970637429656405214 + 18967747941308849394424837233360966669121458757280281874048114287796161191538 * u))), master_map: MasterMap { id: 2930681509702876756 }, reliability_config: None

  2024-06-29T00:29:22.081056Z  INFO builder::testing: Before init hotshot
    at builder/src/lib.rs:402

  2024-06-29T00:29:22.119609Z  INFO builder::testing: After init hotshot
    at builder/src/lib.rs:417

  2024-06-29T00:29:22.119709Z  WARN hotshot_task_impls::helpers: Event: Event { view_number: ViewNumber(0), event: Decide { leaf_chain: [LeafInfo { leaf: Leaf { view_number: ViewNumber(0), justify_qc: SimpleCertificate { data: QuorumData { leaf_commit: Commitment([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], PhantomData<fn(&hotshot_types::data::Leaf<sequencer::SeqTypes>)>) }, vote_commitment: Commitment([187, 245, 119, 137, 16, 15, 21, 51, 50, 14, 67, 240, 228, 38, 111, 46, 117, 179, 14, 185, 231, 151, 175, 43, 180, 151, 216, 73, 166, 140, 255, 104], PhantomData<fn(&hotshot_types::simple_vote::QuorumData<sequencer::SeqTypes>)>), view_number: ViewNumber(0), signatures: None, _pd: PhantomData<(sequencer::SeqTypes, hotshot_types::simple_certificate::SuccessThreshold)> }, parent_commitment: Commitment([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], PhantomData<fn(&hotshot_types::data::Leaf<sequencer::SeqTypes>)>), block_header: Header { chain_config: ResolvableChainConfig { chain_config: Left(ChainConfig { chain_id: ChainId(35353), max_block_size: 10240, base_fee: FeeAmount(0) }) }, height: 0, timestamp: 0, l1_head: 0, l1_finalized: None, payload_commitment: HasherNode([1, 172, 236, 181, 7, 171, 254, 26, 53, 74, 168, 6, 79, 74, 245, 211, 241, 172, 208, 25, 227, 125, 179, 193, 28, 151, 82, 59, 113, 199, 110, 157]), builder_commitment: BuilderCommitment([180, 75, 236, 210, 188, 106, 58, 35, 2, 189, 247, 182, 71, 74, 38, 52, 214, 169, 133, 41, 84, 136, 64, 235, 111, 106, 180, 33, 154, 81, 114, 0]), ns_table: NameSpaceTable { bytes: [0, 0, 0, 0], phantom: PhantomData<u32> }, block_merkle_tree_root: MerkleTreeCommitment { digest: Sha3Node([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]), height: 32, num_leaves: 0 }, fee_merkle_tree_root: MerkleTreeCommitment { digest: Sha3Node([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]), height: 20, num_leaves: 0 }, builder_signature: None, fee_info: FeeInfo { account: FeeAccount(0x0000000000000000000000000000000000000000), amount: FeeAmount(0) } }, upgrade_certificate: None, block_payload: Some(Payload { raw_payload: [], ns_table: NameSpaceTable { bytes: [0, 0, 0, 0], phantom: PhantomData<u32> } }) }, state: ValidatedState { block_merkle_tree: LightWeightMerkleTree { root: Empty, height: 32, num_leaves: 0, _phantom: PhantomData<jf_primitives::merkle_tree::prelude::Sha3Digest> }, fee_merkle_tree: UniversalMerkleTree { root: Empty, height: 20, num_leaves: 0, _phantom: PhantomData<jf_primitives::merkle_tree::prelude::Sha3Digest> } }, delta: Some(Delta { fees_delta: {} }), vid_share: None }], qc: SimpleCertificate { data: QuorumData { leaf_commit: Commitment([121, 160, 70, 40, 95, 62, 151, 15, 173, 219, 93, 112, 196, 186, 187, 181, 197, 196, 18, 166, 156, 110, 240, 213, 112, 81, 81, 245, 160, 157, 116, 237], PhantomData<fn(&hotshot_types::data::Leaf<sequencer::SeqTypes>)>) }, vote_commitment: Commitment([211, 230, 113, 54, 3, 18, 177, 65, 15, 120, 96, 221, 171, 159, 147, 100, 79, 15, 14, 198, 48, 128, 142, 14, 217, 245, 70, 48, 170, 40, 63, 206], PhantomData<fn(&hotshot_types::simple_vote::QuorumData<sequencer::SeqTypes>)>), view_number: ViewNumber(0), signatures: None, _pd: PhantomData<(sequencer::SeqTypes, hotshot_types::simple_certificate::SuccessThreshold)> }, block_size: None } }
 Sending failed, event stream probably shutdown
    at /home/leloup/.cargo-nix/git/checkouts/hotshot-0ac703037ea89b98/f02711d/crates/task-impls/src/helpers.rs:34

  2024-06-29T00:29:22.119803Z  WARN hotshot_task_impls::helpers: Event: Event { view_number: ViewNumber(0), event: Decide { leaf_chain: [LeafInfo { leaf: Leaf { view_number: ViewNumber(0), justify_qc: SimpleCertificate { data: QuorumData { leaf_commit: Commitment([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], PhantomData<fn(&hotshot_types::data::Leaf<sequencer::SeqTypes>)>) }, vote_commitment: Commitment([187, 245, 119, 137, 16, 15, 21, 51, 50, 14, 67, 240, 228, 38, 111, 46, 117, 179, 14, 185, 231, 151, 175, 43, 180, 151, 216, 73, 166, 140, 255, 104], PhantomData<fn(&hotshot_types::simple_vote::QuorumData<sequencer::SeqTypes>)>), view_number: ViewNumber(0), signatures: None, _pd: PhantomData<(sequencer::SeqTypes, hotshot_types::simple_certificate::SuccessThreshold)> }, parent_commitment: Commitment([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], PhantomData<fn(&hotshot_types::data::Leaf<sequencer::SeqTypes>)>), block_header: Header { chain_config: ResolvableChainConfig { chain_config: Left(ChainConfig { chain_id: ChainId(35353), max_block_size: 10240, base_fee: FeeAmount(0) }) }, height: 0, timestamp: 0, l1_head: 0, l1_finalized: None, payload_commitment: HasherNode([1, 172, 236, 181, 7, 171, 254, 26, 53, 74, 168, 6, 79, 74, 245, 211, 241, 172, 208, 25, 227, 125, 179, 193, 28, 151, 82, 59, 113, 199, 110, 157]), builder_commitment: BuilderCommitment([180, 75, 236, 210, 188, 106, 58, 35, 2, 189, 247, 182, 71, 74, 38, 52, 214, 169, 133, 41, 84, 136, 64, 235, 111, 106, 180, 33, 154, 81, 114, 0]), ns_table: NameSpaceTable { bytes: [0, 0, 0, 0], phantom: PhantomData<u32> }, block_merkle_tree_root: MerkleTreeCommitment { digest: Sha3Node([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]), height: 32, num_leaves: 0 }, fee_merkle_tree_root: MerkleTreeCommitment { digest: Sha3Node([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]), height: 20, num_leaves: 0 }, builder_signature: None, fee_info: FeeInfo { account: FeeAccount(0x0000000000000000000000000000000000000000), amount: FeeAmount(0) } }, upgrade_certificate: None, block_payload: Some(Payload { raw_payload: [], ns_table: NameSpaceTable { bytes: [0, 0, 0, 0], phantom: PhantomData<u32> } }) }, state: ValidatedState { block_merkle_tree: LightWeightMerkleTree { root: Empty, height: 32, num_leaves: 0, _phantom: PhantomData<jf_primitives::merkle_tree::prelude::Sha3Digest> }, fee_merkle_tree: UniversalMerkleTree { root: Empty, height: 20, num_leaves: 0, _phantom: PhantomData<jf_primitives::merkle_tree::prelude::Sha3Digest> } }, delta: Some(Delta { fees_delta: {} }), vid_share: None }], qc: SimpleCertificate { data: QuorumData { leaf_commit: Commitment([121, 160, 70, 40, 95, 62, 151, 15, 173, 219, 93, 112, 196, 186, 187, 181, 197, 196, 18, 166, 156, 110, 240, 213, 112, 81, 81, 245, 160, 157, 116, 237], PhantomData<fn(&hotshot_types::data::Leaf<sequencer::SeqTypes>)>) }, vote_commitment: Commitment([211, 230, 113, 54, 3, 18, 177, 65, 15, 120, 96, 221, 171, 159, 147, 100, 79, 15, 14, 198, 48, 128, 142, 14, 217, 245, 70, 48, 170, 40, 63, 206], PhantomData<fn(&hotshot_types::simple_vote::QuorumData<sequencer::SeqTypes>)>), view_number: ViewNumber(0), signatures: None, _pd: PhantomData<(sequencer::SeqTypes, hotshot_types::simple_certificate::SuccessThreshold)> }, block_size: None } }
 Sending failed, event stream probably shutdown
    at /home/leloup/.cargo-nix/git/checkouts/hotshot-0ac703037ea89b98/f02711d/crates/task-impls/src/helpers.rs:34

  2024-06-29T00:29:22.119878Z  WARN hotshot_task_impls::helpers: Event: Event { view_number: ViewNumber(0), event: Decide { leaf_chain: [LeafInfo { leaf: Leaf { view_number: ViewNumber(0), justify_qc: SimpleCertificate { data: QuorumData { leaf_commit: Commitment([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], PhantomData<fn(&hotshot_types::data::Leaf<sequencer::SeqTypes>)>) }, vote_commitment: Commitment([187, 245, 119, 137, 16, 15, 21, 51, 50, 14, 67, 240, 228, 38, 111, 46, 117, 179, 14, 185, 231, 151, 175, 43, 180, 151, 216, 73, 166, 140, 255, 104], PhantomData<fn(&hotshot_types::simple_vote::QuorumData<sequencer::SeqTypes>)>), view_number: ViewNumber(0), signatures: None, _pd: PhantomData<(sequencer::SeqTypes, hotshot_types::simple_certificate::SuccessThreshold)> }, parent_commitment: Commitment([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], PhantomData<fn(&hotshot_types::data::Leaf<sequencer::SeqTypes>)>), block_header: Header { chain_config: ResolvableChainConfig { chain_config: Left(ChainConfig { chain_id: ChainId(35353), max_block_size: 10240, base_fee: FeeAmount(0) }) }, height: 0, timestamp: 0, l1_head: 0, l1_finalized: None, payload_commitment: HasherNode([1, 172, 236, 181, 7, 171, 254, 26, 53, 74, 168, 6, 79, 74, 245, 211, 241, 172, 208, 25, 227, 125, 179, 193, 28, 151, 82, 59, 113, 199, 110, 157]), builder_commitment: BuilderCommitment([180, 75, 236, 210, 188, 106, 58, 35, 2, 189, 247, 182, 71, 74, 38, 52, 214, 169, 133, 41, 84, 136, 64, 235, 111, 106, 180, 33, 154, 81, 114, 0]), ns_table: NameSpaceTable { bytes: [0, 0, 0, 0], phantom: PhantomData<u32> }, block_merkle_tree_root: MerkleTreeCommitment { digest: Sha3Node([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]), height: 32, num_leaves: 0 }, fee_merkle_tree_root: MerkleTreeCommitment { digest: Sha3Node([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]), height: 20, num_leaves: 0 }, builder_signature: None, fee_info: FeeInfo { account: FeeAccount(0x0000000000000000000000000000000000000000), amount: FeeAmount(0) } }, upgrade_certificate: None, block_payload: Some(Payload { raw_payload: [], ns_table: NameSpaceTable { bytes: [0, 0, 0, 0], phantom: PhantomData<u32> } }) }, state: ValidatedState { block_merkle_tree: LightWeightMerkleTree { root: Empty, height: 32, num_leaves: 0, _phantom: PhantomData<jf_primitives::merkle_tree::prelude::Sha3Digest> }, fee_merkle_tree: UniversalMerkleTree { root: Empty, height: 20, num_leaves: 0, _phantom: PhantomData<jf_primitives::merkle_tree::prelude::Sha3Digest> } }, delta: Some(Delta { fees_delta: {} }), vid_share: None }], qc: SimpleCertificate { data: QuorumData { leaf_commit: Commitment([121, 160, 70, 40, 95, 62, 151, 15, 173, 219, 93, 112, 196, 186, 187, 181, 197, 196, 18, 166, 156, 110, 240, 213, 112, 81, 81, 245, 160, 157, 116, 237], PhantomData<fn(&hotshot_types::data::Leaf<sequencer::SeqTypes>)>) }, vote_commitment: Commitment([211, 230, 113, 54, 3, 18, 177, 65, 15, 120, 96, 221, 171, 159, 147, 100, 79, 15, 14, 198, 48, 128, 142, 14, 217, 245, 70, 48, 170, 40, 63, 206], PhantomData<fn(&hotshot_types::simple_vote::QuorumData<sequencer::SeqTypes>)>), view_number: ViewNumber(0), signatures: None, _pd: PhantomData<(sequencer::SeqTypes, hotshot_types::simple_certificate::SuccessThreshold)> }, block_size: None } }
 Sending failed, event stream probably shutdown
    at /home/leloup/.cargo-nix/git/checkouts/hotshot-0ac703037ea89b98/f02711d/crates/task-impls/src/helpers.rs:34

  2024-06-29T00:29:22.119977Z  WARN hotshot_task_impls::helpers: Event: Event { view_number: ViewNumber(0), event: Decide { leaf_chain: [LeafInfo { leaf: Leaf { view_number: ViewNumber(0), justify_qc: SimpleCertificate { data: QuorumData { leaf_commit: Commitment([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], PhantomData<fn(&hotshot_types::data::Leaf<sequencer::SeqTypes>)>) }, vote_commitment: Commitment([187, 245, 119, 137, 16, 15, 21, 51, 50, 14, 67, 240, 228, 38, 111, 46, 117, 179, 14, 185, 231, 151, 175, 43, 180, 151, 216, 73, 166, 140, 255, 104], PhantomData<fn(&hotshot_types::simple_vote::QuorumData<sequencer::SeqTypes>)>), view_number: ViewNumber(0), signatures: None, _pd: PhantomData<(sequencer::SeqTypes, hotshot_types::simple_certificate::SuccessThreshold)> }, parent_commitment: Commitment([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], PhantomData<fn(&hotshot_types::data::Leaf<sequencer::SeqTypes>)>), block_header: Header { chain_config: ResolvableChainConfig { chain_config: Left(ChainConfig { chain_id: ChainId(35353), max_block_size: 10240, base_fee: FeeAmount(0) }) }, height: 0, timestamp: 0, l1_head: 0, l1_finalized: None, payload_commitment: HasherNode([1, 172, 236, 181, 7, 171, 254, 26, 53, 74, 168, 6, 79, 74, 245, 211, 241, 172, 208, 25, 227, 125, 179, 193, 28, 151, 82, 59, 113, 199, 110, 157]), builder_commitment: BuilderCommitment([180, 75, 236, 210, 188, 106, 58, 35, 2, 189, 247, 182, 71, 74, 38, 52, 214, 169, 133, 41, 84, 136, 64, 235, 111, 106, 180, 33, 154, 81, 114, 0]), ns_table: NameSpaceTable { bytes: [0, 0, 0, 0], phantom: PhantomData<u32> }, block_merkle_tree_root: MerkleTreeCommitment { digest: Sha3Node([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]), height: 32, num_leaves: 0 }, fee_merkle_tree_root: MerkleTreeCommitment { digest: Sha3Node([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]), height: 20, num_leaves: 0 }, builder_signature: None, fee_info: FeeInfo { account: FeeAccount(0x0000000000000000000000000000000000000000), amount: FeeAmount(0) } }, upgrade_certificate: None, block_payload: Some(Payload { raw_payload: [], ns_table: NameSpaceTable { bytes: [0, 0, 0, 0], phantom: PhantomData<u32> } }) }, state: ValidatedState { block_merkle_tree: LightWeightMerkleTree { root: Empty, height: 32, num_leaves: 0, _phantom: PhantomData<jf_primitives::merkle_tree::prelude::Sha3Digest> }, fee_merkle_tree: UniversalMerkleTree { root: Empty, height: 20, num_leaves: 0, _phantom: PhantomData<jf_primitives::merkle_tree::prelude::Sha3Digest> } }, delta: Some(Delta { fees_delta: {} }), vid_share: None }], qc: SimpleCertificate { data: QuorumData { leaf_commit: Commitment([121, 160, 70, 40, 95, 62, 151, 15, 173, 219, 93, 112, 196, 186, 187, 181, 197, 196, 18, 166, 156, 110, 240, 213, 112, 81, 81, 245, 160, 157, 116, 237], PhantomData<fn(&hotshot_types::data::Leaf<sequencer::SeqTypes>)>) }, vote_commitment: Commitment([211, 230, 113, 54, 3, 18, 177, 65, 15, 120, 96, 221, 171, 159, 147, 100, 79, 15, 14, 198, 48, 128, 142, 14, 217, 245, 70, 48, 170, 40, 63, 206], PhantomData<fn(&hotshot_types::simple_vote::QuorumData<sequencer::SeqTypes>)>), view_number: ViewNumber(0), signatures: None, _pd: PhantomData<(sequencer::SeqTypes, hotshot_types::simple_certificate::SuccessThreshold)> }, block_size: None } }
 Sending failed, event stream probably shutdown
    at /home/leloup/.cargo-nix/git/checkouts/hotshot-0ac703037ea89b98/f02711d/crates/task-impls/src/helpers.rs:34

  2024-06-29T00:29:22.120066Z  WARN hotshot_task_impls::helpers: Event: Event { view_number: ViewNumber(0), event: Decide { leaf_chain: [LeafInfo { leaf: Leaf { view_number: ViewNumber(0), justify_qc: SimpleCertificate { data: QuorumData { leaf_commit: Commitment([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], PhantomData<fn(&hotshot_types::data::Leaf<sequencer::SeqTypes>)>) }, vote_commitment: Commitment([187, 245, 119, 137, 16, 15, 21, 51, 50, 14, 67, 240, 228, 38, 111, 46, 117, 179, 14, 185, 231, 151, 175, 43, 180, 151, 216, 73, 166, 140, 255, 104], PhantomData<fn(&hotshot_types::simple_vote::QuorumData<sequencer::SeqTypes>)>), view_number: ViewNumber(0), signatures: None, _pd: PhantomData<(sequencer::SeqTypes, hotshot_types::simple_certificate::SuccessThreshold)> }, parent_commitment: Commitment([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], PhantomData<fn(&hotshot_types::data::Leaf<sequencer::SeqTypes>)>), block_header: Header { chain_config: ResolvableChainConfig { chain_config: Left(ChainConfig { chain_id: ChainId(35353), max_block_size: 10240, base_fee: FeeAmount(0) }) }, height: 0, timestamp: 0, l1_head: 0, l1_finalized: None, payload_commitment: HasherNode([1, 172, 236, 181, 7, 171, 254, 26, 53, 74, 168, 6, 79, 74, 245, 211, 241, 172, 208, 25, 227, 125, 179, 193, 28, 151, 82, 59, 113, 199, 110, 157]), builder_commitment: BuilderCommitment([180, 75, 236, 210, 188, 106, 58, 35, 2, 189, 247, 182, 71, 74, 38, 52, 214, 169, 133, 41, 84, 136, 64, 235, 111, 106, 180, 33, 154, 81, 114, 0]), ns_table: NameSpaceTable { bytes: [0, 0, 0, 0], phantom: PhantomData<u32> }, block_merkle_tree_root: MerkleTreeCommitment { digest: Sha3Node([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]), height: 32, num_leaves: 0 }, fee_merkle_tree_root: MerkleTreeCommitment { digest: Sha3Node([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]), height: 20, num_leaves: 0 }, builder_signature: None, fee_info: FeeInfo { account: FeeAccount(0x0000000000000000000000000000000000000000), amount: FeeAmount(0) } }, upgrade_certificate: None, block_payload: Some(Payload { raw_payload: [], ns_table: NameSpaceTable { bytes: [0, 0, 0, 0], phantom: PhantomData<u32> } }) }, state: ValidatedState { block_merkle_tree: LightWeightMerkleTree { root: Empty, height: 32, num_leaves: 0, _phantom: PhantomData<jf_primitives::merkle_tree::prelude::Sha3Digest> }, fee_merkle_tree: UniversalMerkleTree { root: Empty, height: 20, num_leaves: 0, _phantom: PhantomData<jf_primitives::merkle_tree::prelude::Sha3Digest> } }, delta: Some(Delta { fees_delta: {} }), vid_share: None }], qc: SimpleCertificate { data: QuorumData { leaf_commit: Commitment([121, 160, 70, 40, 95, 62, 151, 15, 173, 219, 93, 112, 196, 186, 187, 181, 197, 196, 18, 166, 156, 110, 240, 213, 112, 81, 81, 245, 160, 157, 116, 237], PhantomData<fn(&hotshot_types::data::Leaf<sequencer::SeqTypes>)>) }, vote_commitment: Commitment([211, 230, 113, 54, 3, 18, 177, 65, 15, 120, 96, 221, 171, 159, 147, 100, 79, 15, 14, 198, 48, 128, 142, 14, 217, 245, 70, 48, 170, 40, 63, 206], PhantomData<fn(&hotshot_types::simple_vote::QuorumData<sequencer::SeqTypes>)>), view_number: ViewNumber(0), signatures: None, _pd: PhantomData<(sequencer::SeqTypes, hotshot_types::simple_certificate::SuccessThreshold)> }, block_size: None } }
 Sending failed, event stream probably shutdown
    at /home/leloup/.cargo-nix/git/checkouts/hotshot-0ac703037ea89b98/f02711d/crates/task-impls/src/helpers.rs:34

  2024-06-29T00:29:22.120138Z  WARN hotshot_task_impls::helpers: Event: Event { view_number: ViewNumber(0), event: Decide { leaf_chain: [LeafInfo { leaf: Leaf { view_number: ViewNumber(0), justify_qc: SimpleCertificate { data: QuorumData { leaf_commit: Commitment([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], PhantomData<fn(&hotshot_types::data::Leaf<sequencer::SeqTypes>)>) }, vote_commitment: Commitment([187, 245, 119, 137, 16, 15, 21, 51, 50, 14, 67, 240, 228, 38, 111, 46, 117, 179, 14, 185, 231, 151, 175, 43, 180, 151, 216, 73, 166, 140, 255, 104], PhantomData<fn(&hotshot_types::simple_vote::QuorumData<sequencer::SeqTypes>)>), view_number: ViewNumber(0), signatures: None, _pd: PhantomData<(sequencer::SeqTypes, hotshot_types::simple_certificate::SuccessThreshold)> }, parent_commitment: Commitment([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], PhantomData<fn(&hotshot_types::data::Leaf<sequencer::SeqTypes>)>), block_header: Header { chain_config: ResolvableChainConfig { chain_config: Left(ChainConfig { chain_id: ChainId(35353), max_block_size: 10240, base_fee: FeeAmount(0) }) }, height: 0, timestamp: 0, l1_head: 0, l1_finalized: None, payload_commitment: HasherNode([1, 172, 236, 181, 7, 171, 254, 26, 53, 74, 168, 6, 79, 74, 245, 211, 241, 172, 208, 25, 227, 125, 179, 193, 28, 151, 82, 59, 113, 199, 110, 157]), builder_commitment: BuilderCommitment([180, 75, 236, 210, 188, 106, 58, 35, 2, 189, 247, 182, 71, 74, 38, 52, 214, 169, 133, 41, 84, 136, 64, 235, 111, 106, 180, 33, 154, 81, 114, 0]), ns_table: NameSpaceTable { bytes: [0, 0, 0, 0], phantom: PhantomData<u32> }, block_merkle_tree_root: MerkleTreeCommitment { digest: Sha3Node([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]), height: 32, num_leaves: 0 }, fee_merkle_tree_root: MerkleTreeCommitment { digest: Sha3Node([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]), height: 20, num_leaves: 0 }, builder_signature: None, fee_info: FeeInfo { account: FeeAccount(0x0000000000000000000000000000000000000000), amount: FeeAmount(0) } }, upgrade_certificate: None, block_payload: Some(Payload { raw_payload: [], ns_table: NameSpaceTable { bytes: [0, 0, 0, 0], phantom: PhantomData<u32> } }) }, state: ValidatedState { block_merkle_tree: LightWeightMerkleTree { root: Empty, height: 32, num_leaves: 0, _phantom: PhantomData<jf_primitives::merkle_tree::prelude::Sha3Digest> }, fee_merkle_tree: UniversalMerkleTree { root: Empty, height: 20, num_leaves: 0, _phantom: PhantomData<jf_primitives::merkle_tree::prelude::Sha3Digest> } }, delta: Some(Delta { fees_delta: {} }), vid_share: None }], qc: SimpleCertificate { data: QuorumData { leaf_commit: Commitment([121, 160, 70, 40, 95, 62, 151, 15, 173, 219, 93, 112, 196, 186, 187, 181, 197, 196, 18, 166, 156, 110, 240, 213, 112, 81, 81, 245, 160, 157, 116, 237], PhantomData<fn(&hotshot_types::data::Leaf<sequencer::SeqTypes>)>) }, vote_commitment: Commitment([211, 230, 113, 54, 3, 18, 177, 65, 15, 120, 96, 221, 171, 159, 147, 100, 79, 15, 14, 198, 48, 128, 142, 14, 217, 245, 70, 48, 170, 40, 63, 206], PhantomData<fn(&hotshot_types::simple_vote::QuorumData<sequencer::SeqTypes>)>), view_number: ViewNumber(0), signatures: None, _pd: PhantomData<(sequencer::SeqTypes, hotshot_types::simple_certificate::SuccessThreshold)> }, block_size: None } }
 Sending failed, event stream probably shutdown
    at /home/leloup/.cargo-nix/git/checkouts/hotshot-0ac703037ea89b98/f02711d/crates/task-impls/src/helpers.rs:34

  2024-06-29T00:29:22.120798Z  INFO builder::non_permissioned: Builder client connecting to hotshot events API at http://localhost:22197/
    at builder/src/non_permissioned.rs:173

  2024-06-29T00:29:22.157910Z  INFO builder::non_permissioned: Builder client connected to the hotshot events api
    at builder/src/non_permissioned.rs:181

  2024-06-29T00:29:22.158361Z  INFO builder::non_permissioned: Builder client subscribed to hotshot events
    at builder/src/non_permissioned.rs:190

  2024-06-29T00:29:22.158384Z  INFO builder::non_permissioned: Builder init finished
    at builder/src/non_permissioned.rs:204

  2024-06-29T00:29:25.223748Z  INFO builder::non_permissioned::test: Received Available Blocks: [AvailableBlockInfo { block_hash: BuilderCommitment([180, 75, 236, 210, 188, 106, 58, 35, 2, 189, 247, 182, 71, 74, 38, 52, 214, 169, 133, 41, 84, 136, 64, 235, 111, 106, 180, 33, 154, 81, 114, 0]), block_size: 0, offered_fee: 0, signature: Signature { r: 83311318540445447550979940037232861987344597456561040987355507747112166946694, s: 21057822354871850460979274612092207260196961602466225199485742975525966003742, v: 28 }, sender: FeeAccount(0x83afa9893b9c82cf58970738437f123e95ac5cd0), _phantom: PhantomData<sequencer::SeqTypes> }]
    at builder/src/non_permissioned.rs:345

  2024-06-29T00:29:25.271663Z  INFO builder::non_permissioned::test: Received Block Data: AvailableBlockData { block_payload: Payload { raw_payload: [], ns_table: NameSpaceTable { bytes: [0, 0, 0, 0], phantom: PhantomData<u32> } }, metadata: NameSpaceTable { bytes: [0, 0, 0, 0], phantom: PhantomData<u32> }, signature: Signature { r: 10813344286108501359340646365401810168886561610230151000105780348420940520594, s: 53516587595141379915088273772566728166713703079011835487047140335799940928507, v: 28 }, sender: FeeAccount(0x83afa9893b9c82cf58970738437f123e95ac5cd0) }
    at builder/src/non_permissioned.rs:372

The application panicked (crashed).
Message:  Error getting claiming block header BlockClaim { source: Error { message: "Block Header not found" }, resource: "BUILDER_COMMITMENT~tEvs0rxqOiMCvfe2R0omNNaphSlUiEDrb2q0IZpRcgA_" }
Location: builder/src/non_permissioned.rs:393

  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ BACKTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
                                ⋮ 7 frames hidden ⋮                               
   8: builder::non_permissioned::test::test_non_permissioned_builder::{{closure}}::h5f4914e90b355cf1
      at <unknown source file>:<unknown line>
   9: <futures_lite::future::Or<F1,F2> as core::future::future::Future>::poll::h2f25558006e3f9ff
      at <unknown source file>:<unknown line>
  10: async_io::driver::block_on::he2ef5851cbceca79
      at <unknown source file>:<unknown line>
  11: async_global_executor::executor::block_on::h4674899828a310c3
      at <unknown source file>:<unknown line>
  12: async_std::task::task_locals_wrapper::TaskLocalsWrapper::set_current::h86b1836b6aad279f
      at <unknown source file>:<unknown line>
  13: async_std::task::builder::Builder::blocking::h573cb934fae52b84
      at <unknown source file>:<unknown line>
  14: core::ops::function::FnOnce::call_once::h7bc7d8e9b134c304
      at <unknown source file>:<unknown line>
  15: test::__rust_begin_short_backtrace::hda6790495e130382
      at <unknown source file>:<unknown line>
  16: test::run_test::{{closure}}::hee0b72cbce9a714c
      at <unknown source file>:<unknown line>
  17: std::sys_common::backtrace::__rust_begin_short_backtrace::h41308d0c272d8ad4
      at <unknown source file>:<unknown line>
  18: core::ops::function::FnOnce::call_once{{vtable.shim}}::h984c7e6d2d8f5c80
      at <unknown source file>:<unknown line>
  19: std::sys::pal::unix::thread::Thread::new::thread_start::h3631815ad38387d6
      at <unknown source file>:<unknown line>
  20: start_thread<unknown>
      at <unknown source file>:<unknown line>
  21: __clone3<unknown>
      at <unknown source file>:<unknown line>

Run with COLORBT_SHOW_HIDDEN=1 environment variable to disable frame filtering.
Run with RUST_BACKTRACE=full to include source snippets.

failures:
    non_permissioned::test::test_non_permissioned_builder

test result: FAILED. 1 passed; 1 failed; 1 ignored; 0 measured; 0 filtered out; finished in 3.74s

error: test failed, to rerun pass `-p builder --lib`
error: Recipe `test` failed on line 31 with exit code 101

philippecamacho commented 3 months ago

@alxiong I don't think this PR is the cause of the issue. After merging main into this branch (See #1655), all the tests run fine for me locally.

[EDIT] Well it looks like there are some intermittent failures

The application panicked (crashed).
Message:  failed to start postgres docker: docker: Error response from daemon: driver failed programming external connectivity on endpoint optimistic_haibt (f0a6eeef273f5510e7a56683e8e2185ac5fbb3c4a56e898f2c8f64de0a598745): failed to bind port 0.0.0.0:15403/tcp: Error starting userland proxy: listen tcp4 0.0.0.0:15403: bind: address already in use.

Location: /home/leloup/.cargo-nix/git/checkouts/hotshot-query-service-99b1feaafbec0e04/bd03a5d/src/data_source/storage/sql.rs:3047

  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ BACKTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
                                ⋮ 7 frames hidden ⋮                               
   8: sequencer::api::sql::impl_testable_data_source::<impl sequencer::api::data_source::testing::TestableSequencerDataSource for hotshot_query_service::data_source::fetching::FetchingDataSource<sequencer::SeqTypes,hotshot_query_service::data_source::storage::sql::SqlStorage,hotshot_query_service::fetching::provider::any::AnyProvider<sequencer::SeqTypes>>>::create_storage::{{closure}}::hffbf51f003a055e5
      at <unknown source file>:<unknown line>
   9: async_std::task::task_locals_wrapper::TaskLocalsWrapper::set_current::h92d746be65f11580
      at <unknown source file>:<unknown line>
  10: <futures_lite::future::Or<F1,F2> as core::future::future::Future>::poll::h2f7182a9501194b8
      at <unknown source file>:<unknown line>
  11: async_io::driver::block_on::hf6a036aa74290c30
      at <unknown source file>:<unknown line>
  12: async_global_executor::executor::block_on::ha0fbecf360ec5be7
      at <unknown source file>:<unknown line>
  13: async_std::task::task_locals_wrapper::TaskLocalsWrapper::set_current::hd5d641f10d3d5cc2
      at <unknown source file>:<unknown line>
  14: async_std::task::builder::Builder::blocking::h32aec6ab6a62adef
      at <unknown source file>:<unknown line>
  15: core::ops::function::FnOnce::call_once::he3f33275d91faf7e
      at <unknown source file>:<unknown line>
  16: test::__rust_begin_short_backtrace::hda6790495e130382
      at <unknown source file>:<unknown line>
  17: test::run_test::{{closure}}::hee0b72cbce9a714c
      at <unknown source file>:<unknown line>
  18: std::sys_common::backtrace::__rust_begin_short_backtrace::h41308d0c272d8ad4
      at <unknown source file>:<unknown line>
  19: core::ops::function::FnOnce::call_once{{vtable.shim}}::h984c7e6d2d8f5c80
      at <unknown source file>:<unknown line>
  20: std::sys::pal::unix::thread::Thread::new::thread_start::h3631815ad38387d6
      at <unknown source file>:<unknown line>
  21: start_thread<unknown>
      at <unknown source file>:<unknown line>
  22: __clone3<unknown>
      at <unknown source file>:<unknown line>

Run with COLORBT_SHOW_HIDDEN=1 environment variable to disable frame filtering.
Run with RUST_BACKTRACE=full to include source snippets.

failures:
    api::sql::generic_tests::catchup_test_with_query_module

test result: FAILED. 99 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 86.17s

error: test failed, to rerun pass `-p sequencer --lib`
error: Recipe `test` failed on line 34 with exit code 101

alxiong commented 3 months ago

Thanks @philippecamacho for the help!!

EspressoSystems / espresso-sequencer

fix: bug on lagrange coeff computation #1639

This PR: