search

stratum-mining / stratum

stratum
https://stratumprotocol.org
Other
223 stars 127 forks source link

Add integration test for auth failure between `mining_device` and a `pool` #1164

Closed jbesraa closed 1 month ago

jbesraa commented 2 months ago

Blocked by #1163 #1066 #1155 #1156 Resolves #1165

Note that majority of the commits here will be hopefully merged in the blocking PRs mentioned above. This PR should end up with the last three commits.

This should make the following test redundant https://github.com/stratum-mining/stratum/blob/main/test/message-generator/test/pool-sri-test-reject-auth/pool-sri-test-reject-auth.json

github-actions[bot] commented 2 months ago

🐰 Bencher Report

Branch1164/merge
Testbedsv2
Click to view all benchmark results
BenchmarkLatencyBenchmark Result
nanoseconds (ns)
(Result Δ%)		Upper Boundary
nanoseconds (ns)
(Limit %)
client_sv2_handle_message_common📈 view plot
🚷 view threshold		44.64
(-0.15%)		46.67
(95.64%)
client_sv2_handle_message_mining📈 view plot
🚷 view threshold		72.78
(-3.23%)		92.52
(78.67%)
client_sv2_mining_message_submit_standard📈 view plot
🚷 view threshold		14.71
(+0.21%)		14.81
(99.29%)
client_sv2_mining_message_submit_standard_serialize📈 view plot
🚷 view threshold		264.40
(-1.35%)		291.02
(90.85%)
client_sv2_mining_message_submit_standard_serialize_deserialize📈 view plot
🚷 view threshold		581.41
(-1.84%)		649.89
(89.46%)
client_sv2_open_channel📈 view plot
🚷 view threshold		173.70
(+4.57%)		178.27
(97.43%)
client_sv2_open_channel_serialize📈 view plot
🚷 view threshold		273.93
(-1.04%)		306.98
(89.23%)
client_sv2_open_channel_serialize_deserialize📈 view plot
🚷 view threshold		370.75
(-1.78%)		457.71
(81.00%)
client_sv2_setup_connection📈 view plot
🚷 view threshold		160.77
(-0.76%)		185.60
(86.62%)
client_sv2_setup_connection_serialize📈 view plot
🚷 view threshold		467.71
(+2.33%)		566.81
(82.52%)
client_sv2_setup_connection_serialize_deserialize📈 view plot
🚷 view threshold		1,034.10
(+3.50%)		1,163.31
(88.89%)
🐰 View full continuous benchmarking report in Bencher
github-actions[bot] commented 2 months ago

🐰 Bencher Report

Branch1164/merge
Testbedsv1
Click to view all benchmark results
BenchmarkEstimated CyclesBenchmark Result
estimated cycles
(Result Δ%)		Upper Boundary
estimated cycles
(Limit %)		InstructionsBenchmark Result
instructions
(Result Δ%)		Upper Boundary
instructions
(Limit %)		L1 AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)		L2 AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)		RAM AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)
get_authorize📈 view plot
🚷 view threshold		8,422.00
(-0.27%)		8,791.61
(95.80%)		📈 view plot
🚷 view threshold		3,774.00
(+0.69%)		3,881.48
(97.23%)		📈 view plot
🚷 view threshold		5,297.00
(+0.76%)		5,440.11
(97.37%)		📈 view plot
🚷 view threshold		2.00
(-71.24%)		11.81
(16.93%)		📈 view plot
🚷 view threshold		89.00
(-1.21%)		95.29
(93.40%)
get_submit📈 view plot
🚷 view threshold		95,335.00
(-0.15%)		96,198.69
(99.10%)		📈 view plot
🚷 view threshold		59,467.00
(-0.00%)		59,794.75
(99.45%)		📈 view plot
🚷 view threshold		85,415.00
(+0.00%)		85,872.23
(99.47%)		📈 view plot
🚷 view threshold		38.00
(-20.38%)		71.78
(52.94%)		📈 view plot
🚷 view threshold		278.00
(-0.97%)		289.96
(95.88%)
get_subscribe📈 view plot
🚷 view threshold		8,053.00
(+0.55%)		8,359.46
(96.33%)		📈 view plot
🚷 view threshold		2,869.00
(+1.13%)		2,964.25
(96.79%)		📈 view plot
🚷 view threshold		4,013.00
(+1.19%)		4,136.40
(97.02%)		📈 view plot
🚷 view threshold		10.00
(-22.47%)		23.21
(43.09%)		📈 view plot
🚷 view threshold		114.00
(+0.28%)		119.61
(95.31%)
serialize_authorize📈 view plot
🚷 view threshold		12,273.00
(+0.15%)		12,633.26
(97.15%)		📈 view plot
🚷 view threshold		5,345.00
(+0.48%)		5,452.48
(98.03%)		📈 view plot
🚷 view threshold		7,458.00
(+0.53%)		7,602.39
(98.10%)		📈 view plot
🚷 view threshold		4.00
(-56.39%)		15.18
(26.35%)		📈 view plot
🚷 view threshold		137.00
(+0.11%)		143.64
(95.38%)
serialize_deserialize_authorize📈 view plot
🚷 view threshold		24,699.00
(+0.37%)		25,131.57
(98.28%)		📈 view plot
🚷 view threshold		9,896.00
(+0.01%)		10,052.69
(98.44%)		📈 view plot
🚷 view threshold		13,974.00
(+0.05%)		14,196.80
(98.43%)		📈 view plot
🚷 view threshold		31.00
(-11.70%)		44.58
(69.54%)		📈 view plot
🚷 view threshold		302.00
(+1.00%)		312.99
(96.49%)
serialize_deserialize_handle_authorize📈 view plot
🚷 view threshold		30,328.00
(+0.23%)		30,711.04
(98.75%)		📈 view plot
🚷 view threshold		12,099.00
(+0.08%)		12,226.96
(98.95%)		📈 view plot
🚷 view threshold		17,138.00
(+0.12%)		17,308.00
(99.02%)		📈 view plot
🚷 view threshold		55.00
(-2.11%)		68.57
(80.21%)		📈 view plot
🚷 view threshold		369.00
(+0.42%)		380.41
(97.00%)
serialize_deserialize_handle_submit📈 view plot
🚷 view threshold		126,476.00
(+0.02%)		127,166.27
(99.46%)		📈 view plot
🚷 view threshold		73,280.00
(+0.01%)		73,670.59
(99.47%)		📈 view plot
🚷 view threshold		105,056.00
(+0.01%)		105,614.71
(99.47%)		📈 view plot
🚷 view threshold		105.00
(-4.76%)		142.28
(73.80%)		📈 view plot
🚷 view threshold		597.00
(+0.15%)		604.61
(98.74%)
serialize_deserialize_handle_subscribe📈 view plot
🚷 view threshold		27,980.00
(+0.98%)		28,571.58
(97.93%)		📈 view plot
🚷 view threshold		9,687.00
(+0.42%)		9,779.75
(99.05%)		📈 view plot
🚷 view threshold		13,705.00
(+0.44%)		13,830.58
(99.09%)		📈 view plot
🚷 view threshold		62.00
(-2.76%)		75.87
(81.72%)		📈 view plot
🚷 view threshold		399.00
(+1.60%)		415.68
(95.99%)
serialize_deserialize_submit📈 view plot
🚷 view threshold		115,211.00
(+0.03%)		115,885.34
(99.42%)		📈 view plot
🚷 view threshold		68,057.00
(-0.04%)		68,470.14
(99.40%)		📈 view plot
🚷 view threshold		97,656.00
(-0.04%)		98,279.16
(99.37%)		📈 view plot
🚷 view threshold		60.00
(-6.22%)		81.88
(73.28%)		📈 view plot
🚷 view threshold		493.00
(+0.56%)		499.11
(98.78%)
serialize_deserialize_subscribe📈 view plot
🚷 view threshold		23,395.00
(+1.14%)		23,985.22
(97.54%)		📈 view plot
🚷 view threshold		8,239.00
(+0.47%)		8,335.13
(98.85%)		📈 view plot
🚷 view threshold		11,610.00
(+0.51%)		11,737.09
(98.92%)		📈 view plot
🚷 view threshold		33.00
(-12.67%)		45.39
(72.70%)		📈 view plot
🚷 view threshold		332.00
(+2.00%)		347.55
(95.53%)
serialize_submit📈 view plot
🚷 view threshold		99,804.00
(-0.07%)		100,540.17
(99.27%)		📈 view plot
🚷 view threshold		61,511.00
(-0.00%)		61,843.46
(99.46%)		📈 view plot
🚷 view threshold		88,254.00
(-0.00%)		88,724.29
(99.47%)		📈 view plot
🚷 view threshold		42.00
(-14.22%)		70.75
(59.36%)		📈 view plot
🚷 view threshold		324.00
(-0.30%)		333.28
(97.22%)
serialize_subscribe📈 view plot
🚷 view threshold		11,458.00
(+0.63%)		11,760.56
(97.43%)		📈 view plot
🚷 view threshold		4,216.00
(+0.77%)		4,311.25
(97.79%)		📈 view plot
🚷 view threshold		5,868.00
(+0.78%)		5,993.30
(97.91%)		📈 view plot
🚷 view threshold		12.00
(-12.18%)		22.27
(53.88%)		📈 view plot
🚷 view threshold		158.00
(+0.63%)		164.53
(96.03%)
🐰 View full continuous benchmarking report in Bencher
github-actions[bot] commented 2 months ago

🐰 Bencher Report

Branch1164/merge
Testbedsv1
Click to view all benchmark results
BenchmarkLatencyBenchmark Result
nanoseconds (ns)
(Result Δ%)		Upper Boundary
nanoseconds (ns)
(Limit %)
client-submit-serialize📈 view plot
🚷 view threshold		6,796.40
(-0.40%)		7,548.55
(90.04%)
client-submit-serialize-deserialize📈 view plot
🚷 view threshold		7,702.80
(-1.64%)		8,755.80
(87.97%)
client-submit-serialize-deserialize-handle/client-submit-serialize-deserialize-handle📈 view plot
🚷 view threshold		8,343.30
(-0.80%)		9,218.10
(90.51%)
client-sv1-authorize-serialize-deserialize-handle/client-sv1-authorize-serialize-deserialize-handle📈 view plot
🚷 view threshold		902.10
(-1.51%)		985.36
(91.55%)
client-sv1-authorize-serialize-deserialize/client-sv1-authorize-serialize-deserialize📈 view plot
🚷 view threshold		706.73
(-1.24%)		764.84
(92.40%)
client-sv1-authorize-serialize/client-sv1-authorize-serialize📈 view plot
🚷 view threshold		250.31
(+0.17%)		263.12
(95.13%)
client-sv1-get-authorize/client-sv1-get-authorize📈 view plot
🚷 view threshold		157.65
(-0.10%)		166.36
(94.76%)
client-sv1-get-submit📈 view plot
🚷 view threshold		6,565.10
(-0.62%)		7,279.14
(90.19%)
client-sv1-get-subscribe/client-sv1-get-subscribe📈 view plot
🚷 view threshold		275.67
(-0.18%)		300.67
(91.69%)
client-sv1-subscribe-serialize-deserialize-handle/client-sv1-subscribe-serialize-deserialize-handle📈 view plot
🚷 view threshold		745.94
(-2.70%)		829.74
(89.90%)
client-sv1-subscribe-serialize-deserialize/client-sv1-subscribe-serialize-deserialize📈 view plot
🚷 view threshold		627.57
(-0.06%)		681.97
(92.02%)
client-sv1-subscribe-serialize/client-sv1-subscribe-serialize📈 view plot
🚷 view threshold		206.00
(-1.40%)		229.04
(89.94%)
🐰 View full continuous benchmarking report in Bencher
github-actions[bot] commented 2 months ago

🐰 Bencher Report

Branch1164/merge
Testbedsv2
🚨 14 ALERTS: Threshold Boundary Limits exceeded!
BenchmarkMeasure
Units		ViewBenchmark Result
(Result Δ%)		Lower Boundary
(Limit %)		Upper Boundary
(Limit %)
client_sv2_handle_message_commonInstructions
instructions		📈 plot
🚨 alert
🚷 threshold		501.00
(+6.11%)		492.83
(101.66%)
client_sv2_handle_message_commonL1 Accesses
accesses		📈 plot
🚨 alert
🚷 threshold		780.00
(+6.22%)		765.33
(101.92%)
client_sv2_mining_message_submit_standardInstructions
instructions		📈 plot
🚨 alert
🚷 threshold		1,778.00
(+1.57%)		1,770.13
(100.44%)
client_sv2_mining_message_submit_standardL1 Accesses
accesses		📈 plot
🚨 alert
🚷 threshold		2,593.00
(+1.53%)		2,583.54
(100.37%)
client_sv2_mining_message_submit_standard_serializeInstructions
instructions		📈 plot
🚨 alert
🚷 threshold		4,722.00
(+0.59%)		4,714.13
(100.17%)
client_sv2_mining_message_submit_standard_serializeL1 Accesses
accesses		📈 plot
🚨 alert
🚷 threshold		6,803.00
(+0.67%)		6,787.43
(100.23%)
client_sv2_open_channelInstructions
instructions		📈 plot
🚨 alert
🚷 threshold		1,489.00
(+1.93%)		1,480.10
(100.60%)
client_sv2_open_channelL1 Accesses
accesses		📈 plot
🚨 alert
🚷 threshold		2,195.00
(+1.74%)		2,188.35
(100.30%)
client_sv2_open_channel_serializeInstructions
instructions		📈 plot
🚨 alert
🚷 threshold		5,092.00
(+0.56%)		5,083.10
(100.18%)
client_sv2_open_channel_serializeL1 Accesses
accesses		📈 plot
🚨 alert
🚷 threshold		7,365.00
(+0.57%)		7,356.23
(100.12%)
client_sv2_setup_connectionInstructions
instructions		📈 plot
🚨 alert
🚷 threshold		1,530.00
(+1.88%)		1,521.10
(100.58%)
client_sv2_setup_connectionL1 Accesses
accesses		📈 plot
🚨 alert
🚷 threshold		2,320.00
(+1.85%)		2,307.93
(100.52%)
client_sv2_setup_connection_serializeInstructions
instructions		📈 plot
🚨 alert
🚷 threshold		5,991.00
(+0.47%)		5,982.10
(100.15%)
client_sv2_setup_connection_serializeL1 Accesses
accesses		📈 plot
🚨 alert
🚷 threshold		8,708.00
(+0.54%)		8,695.82
(100.14%)
Click to view all benchmark results
BenchmarkEstimated CyclesBenchmark Result
estimated cycles
(Result Δ%)		Upper Boundary
estimated cycles
(Limit %)		InstructionsBenchmark Result
instructions
(Result Δ%)		Upper Boundary
instructions
(Limit %)		L1 AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)		L2 AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)		RAM AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)
client_sv2_handle_message_common📈 view plot
🚷 view threshold		2,115.00
(+1.85%)		2,207.71
(95.80%)		📈 view plot
🚨 view alert
🚷 view threshold		501.00
(+6.11%)		492.83
(101.66%)		📈 view plot
🚨 view alert
🚷 view threshold		780.00
(+6.22%)		765.33
(101.92%)		📈 view plot
🚷 view threshold		1.00
(-82.92%)		14.47
(6.91%)		📈 view plot
🚷 view threshold		38.00
(+1.30%)		41.19
(92.27%)
client_sv2_handle_message_mining📈 view plot
🚷 view threshold		8,116.00
(-1.01%)		8,353.03
(97.16%)		📈 view plot
🚷 view threshold		2,137.00
(+0.17%)		2,180.29
(98.01%)		📈 view plot
🚷 view threshold		3,166.00
(+0.38%)		3,229.92
(98.02%)		📈 view plot
🚷 view threshold		31.00
(-15.36%)		45.49
(68.15%)		📈 view plot
🚷 view threshold		137.00
(-1.38%)		142.39
(96.21%)
client_sv2_mining_message_submit_standard📈 view plot
🚷 view threshold		6,283.00
(+0.07%)		6,417.05
(97.91%)		📈 view plot
🚨 view alert
🚷 view threshold		1,778.00
(+1.57%)		1,770.13
(100.44%)		📈 view plot
🚨 view alert
🚷 view threshold		2,593.00
(+1.53%)		2,583.54
(100.37%)		📈 view plot
🚷 view threshold		17.00
(-0.14%)		24.79
(68.57%)		📈 view plot
🚷 view threshold		103.00
(-0.95%)		107.79
(95.56%)
client_sv2_mining_message_submit_standard_serialize📈 view plot
🚷 view threshold		14,698.00
(-0.32%)		15,055.84
(97.62%)		📈 view plot
🚨 view alert
🚷 view threshold		4,722.00
(+0.59%)		4,714.13
(100.17%)		📈 view plot
🚨 view alert
🚷 view threshold		6,803.00
(+0.67%)		6,787.43
(100.23%)		📈 view plot
🚷 view threshold		39.00
(-12.11%)		55.22
(70.62%)		📈 view plot
🚷 view threshold		220.00
(-0.85%)		230.51
(95.44%)
client_sv2_mining_message_submit_standard_serialize_deserialize📈 view plot
🚷 view threshold		27,598.00
(+0.35%)		27,895.94
(98.93%)		📈 view plot
🚷 view threshold		10,613.00
(+0.44%)		10,640.35
(99.74%)		📈 view plot
🚷 view threshold		15,443.00
(+0.44%)		15,483.47
(99.74%)		📈 view plot
🚷 view threshold		79.00
(-3.30%)		90.57
(87.23%)		📈 view plot
🚷 view threshold		336.00
(+0.35%)		345.95
(97.12%)
client_sv2_open_channel📈 view plot
🚷 view threshold		4,555.00
(+2.98%)		4,674.84
(97.44%)		📈 view plot
🚨 view alert
🚷 view threshold		1,489.00
(+1.93%)		1,480.10
(100.60%)		📈 view plot
🚨 view alert
🚷 view threshold		2,195.00
(+1.74%)		2,188.35
(100.30%)		📈 view plot
🚷 view threshold		10.00
(+2.38%)		18.53
(53.97%)		📈 view plot
🚷 view threshold		66.00
(+4.19%)		69.98
(94.31%)
client_sv2_open_channel_serialize📈 view plot
🚷 view threshold		14,180.00
(+0.50%)		14,529.64
(97.59%)		📈 view plot
🚨 view alert
🚷 view threshold		5,092.00
(+0.56%)		5,083.10
(100.18%)		📈 view plot
🚨 view alert
🚷 view threshold		7,365.00
(+0.57%)		7,356.23
(100.12%)		📈 view plot
🚷 view threshold		33.00
(-6.59%)		43.75
(75.43%)		📈 view plot
🚷 view threshold		190.00
(+0.61%)		200.78
(94.63%)
client_sv2_open_channel_serialize_deserialize📈 view plot
🚷 view threshold		22,819.00
(+0.80%)		23,039.50
(99.04%)		📈 view plot
🚷 view threshold		8,055.00
(+0.59%)		8,080.64
(99.68%)		📈 view plot
🚷 view threshold		11,709.00
(+0.51%)		11,757.23
(99.59%)		📈 view plot
🚷 view threshold		80.00
(+8.59%)		85.23
(93.86%)		📈 view plot
🚷 view threshold		306.00
(+0.85%)		314.93
(97.16%)
client_sv2_setup_connection📈 view plot
🚷 view threshold		4,735.00
(+1.00%)		4,786.10
(98.93%)		📈 view plot
🚨 view alert
🚷 view threshold		1,530.00
(+1.88%)		1,521.10
(100.58%)		📈 view plot
🚨 view alert
🚷 view threshold		2,320.00
(+1.85%)		2,307.93
(100.52%)		📈 view plot
🚷 view threshold		7.00
(-23.58%)		15.74
(44.47%)		📈 view plot
🚷 view threshold		68.00
(+0.66%)		70.30
(96.72%)
client_sv2_setup_connection_serialize📈 view plot
🚷 view threshold		16,168.00
(-0.18%)		16,513.52
(97.91%)		📈 view plot
🚨 view alert
🚷 view threshold		5,991.00
(+0.47%)		5,982.10
(100.15%)		📈 view plot
🚨 view alert
🚷 view threshold		8,708.00
(+0.54%)		8,695.82
(100.14%)		📈 view plot
🚷 view threshold		36.00
(-12.88%)		54.96
(65.50%)		📈 view plot
🚷 view threshold		208.00
(-0.69%)		217.84
(95.48%)
client_sv2_setup_connection_serialize_deserialize📈 view plot
🚷 view threshold		35,784.00
(+0.66%)		35,792.70
(99.98%)		📈 view plot
🚷 view threshold		14,883.00
(+0.32%)		14,909.69
(99.82%)		📈 view plot
🚷 view threshold		21,859.00
(+0.31%)		21,913.10
(99.75%)		📈 view plot
🚷 view threshold		90.00
(-4.54%)		116.03
(77.57%)		📈 view plot
🚷 view threshold		385.00
(+1.42%)		385.35
(99.91%)
🐰 View full continuous benchmarking report in Bencher
github-actions[bot] commented 2 months ago

🐰 Bencher Report

Branch1164/merge@013be4ec-af5a-40f0-8b58-76e514ea8d3f
Testbedsv1
Click to view all benchmark results
BenchmarkLatencyBenchmark Result
nanoseconds (ns)
(Result Δ%)		Upper Boundary
nanoseconds (ns)
(Limit %)
client-submit-serialize📈 view plot
🚷 view threshold		6,659.40
(-2.38%)		7,550.91
(88.19%)
client-submit-serialize-deserialize📈 view plot
🚷 view threshold		7,528.60
(-3.83%)		8,763.26
(85.91%)
client-submit-serialize-deserialize-handle/client-submit-serialize-deserialize-handle📈 view plot
🚷 view threshold		8,268.60
(-1.67%)		9,219.35
(89.69%)
client-sv1-authorize-serialize-deserialize-handle/client-sv1-authorize-serialize-deserialize-handle📈 view plot
🚷 view threshold		923.23
(+0.76%)		985.44
(93.69%)
client-sv1-authorize-serialize-deserialize/client-sv1-authorize-serialize-deserialize📈 view plot
🚷 view threshold		720.55
(+0.66%)		764.92
(94.20%)
client-sv1-authorize-serialize/client-sv1-authorize-serialize📈 view plot
🚷 view threshold		248.27
(-0.64%)		263.11
(94.36%)
client-sv1-get-authorize/client-sv1-get-authorize📈 view plot
🚷 view threshold		156.56
(-0.78%)		166.36
(94.11%)
client-sv1-get-submit📈 view plot
🚷 view threshold		6,497.00
(-1.64%)		7,279.97
(89.24%)
client-sv1-get-subscribe/client-sv1-get-subscribe📈 view plot
🚷 view threshold		283.44
(+2.58%)		301.06
(94.15%)
client-sv1-subscribe-serialize-deserialize-handle/client-sv1-subscribe-serialize-deserialize-handle📈 view plot
🚷 view threshold		745.22
(-2.79%)		829.79
(89.81%)
client-sv1-subscribe-serialize-deserialize/client-sv1-subscribe-serialize-deserialize📈 view plot
🚷 view threshold		636.39
(+1.32%)		682.28
(93.27%)
client-sv1-subscribe-serialize/client-sv1-subscribe-serialize📈 view plot
🚷 view threshold		210.44
(+0.69%)		229.07
(91.87%)
🐰 View full continuous benchmarking report in Bencher
jbesraa commented 1 month ago

will close this for now and wait for #1066 to get merged and then maybe re-open this..