search

k1LoW / runn

runn is a package/tool for running operations following a scenario.
https://runn.run
MIT License
417 stars 30 forks source link

Fix test for problem with validators behaving incorrectly when running concurrently #906

Closed k1LoW closed 4 months ago

k1LoW commented 4 months ago

ref: https://github.com/k1LoW/runn/issues/882#issuecomment-2082602173

github-actions[bot] commented 4 months ago

BenchmarkManyRunbooks-4

main (-) #906 (fbb3409) +/-
Number of iterations 1 1 0
Nanoseconds per iteration 3,800,379,994 ns/op 3,773,674,710 ns/op -26,705,284 ns/op
Bytes allocated per iteration 1,793,010,752 B/op 1,792,494,936 B/op -515,816 B/op
Allocs per iteration 21,418,771 allocs/op 21,418,414 allocs/op -357 allocs/op
Metadata | | [main](https://github.com/k1LoW/runn/tree/main) (-) | [#906](https://github.com/k1LoW/runn/pull/906) ([fbb3409](https://github.com/k1LoW/runn/commit/fbb34095b4723cd7576bac621292c481ac5c96e9)) | |------------|----------------------------------------------------:|------------------------------------------------------------------------------------------------------------------------------------------:| | **goos** | linux | linux | | **goarch** | amd64 | amd64 | | **pkg** | github.com/k1LoW/runn | github.com/k1LoW/runn | | **cpu** | AMD EPYC 7763 64-Core Processor | AMD EPYC 7763 64-Core Processor |

BenchmarkOpenAPI3-4

main (-) #906 (fbb3409) +/-
Number of iterations 1 1 0
Nanoseconds per iteration 5,355,452,880 ns/op 5,372,044,926 ns/op 16,592,046 ns/op
Bytes allocated per iteration 2,944,796,912 B/op 2,944,988,056 B/op 191,144 B/op
Allocs per iteration 29,269,073 allocs/op 29,264,430 allocs/op -4,643 allocs/op
Metadata | | [main](https://github.com/k1LoW/runn/tree/main) (-) | [#906](https://github.com/k1LoW/runn/pull/906) ([fbb3409](https://github.com/k1LoW/runn/commit/fbb34095b4723cd7576bac621292c481ac5c96e9)) | |------------|----------------------------------------------------:|------------------------------------------------------------------------------------------------------------------------------------------:| | **goos** | linux | linux | | **goarch** | amd64 | amd64 | | **pkg** | github.com/k1LoW/runn | github.com/k1LoW/runn | | **cpu** | AMD EPYC 7763 64-Core Processor | AMD EPYC 7763 64-Core Processor |

BenchmarkSingleRunbook-4

main (-) #906 (fbb3409) +/-
Number of iterations 18 18 0
Nanoseconds per iteration 66,732,370 ns/op 65,664,797 ns/op -1,067,573 ns/op
Bytes allocated per iteration 60,289,820 B/op 60,263,611 B/op -26,209 B/op
Allocs per iteration 205,875 allocs/op 205,851 allocs/op -24 allocs/op
Metadata | | [main](https://github.com/k1LoW/runn/tree/main) (-) | [#906](https://github.com/k1LoW/runn/pull/906) ([fbb3409](https://github.com/k1LoW/runn/commit/fbb34095b4723cd7576bac621292c481ac5c96e9)) | |------------|----------------------------------------------------:|------------------------------------------------------------------------------------------------------------------------------------------:| | **goos** | linux | linux | | **goarch** | amd64 | amd64 | | **pkg** | github.com/k1LoW/runn | github.com/k1LoW/runn | | **cpu** | AMD EPYC 7763 64-Core Processor | AMD EPYC 7763 64-Core Processor |

Reported by octocov

github-actions[bot] commented 4 months ago

BenchmarkManyRunbooks-4

main (-) #906 (562f8e9) +/-
Number of iterations 1 1 0
Nanoseconds per iteration 3,828,830,318 ns/op 3,988,894,164 ns/op 160,063,846 ns/op
Bytes allocated per iteration 1,792,369,472 B/op 1,792,313,560 B/op -55,912 B/op
Allocs per iteration 21,418,268 allocs/op 21,417,702 allocs/op -566 allocs/op
Metadata | | [main](https://github.com/k1LoW/runn/tree/main) (-) | [#906](https://github.com/k1LoW/runn/pull/906) ([562f8e9](https://github.com/k1LoW/runn/commit/562f8e9e55177a6f52b8674821c77b88740554df)) | |------------|----------------------------------------------------:|------------------------------------------------------------------------------------------------------------------------------------------:| | **goos** | linux | linux | | **goarch** | amd64 | amd64 | | **pkg** | github.com/k1LoW/runn | github.com/k1LoW/runn | | **cpu** | AMD EPYC 7763 64-Core Processor | AMD EPYC 7763 64-Core Processor |

BenchmarkOpenAPI3-4

main (-) #906 (562f8e9) +/-
Number of iterations 1 1 0
Nanoseconds per iteration 5,331,258,656 ns/op 5,105,884,454 ns/op -225,374,202 ns/op
Bytes allocated per iteration 2,944,991,912 B/op 2,944,741,032 B/op -250,880 B/op
Allocs per iteration 29,263,169 allocs/op 29,261,637 allocs/op -1,532 allocs/op
Metadata | | [main](https://github.com/k1LoW/runn/tree/main) (-) | [#906](https://github.com/k1LoW/runn/pull/906) ([562f8e9](https://github.com/k1LoW/runn/commit/562f8e9e55177a6f52b8674821c77b88740554df)) | |------------|----------------------------------------------------:|------------------------------------------------------------------------------------------------------------------------------------------:| | **goos** | linux | linux | | **goarch** | amd64 | amd64 | | **pkg** | github.com/k1LoW/runn | github.com/k1LoW/runn | | **cpu** | AMD EPYC 7763 64-Core Processor | AMD EPYC 7763 64-Core Processor |

BenchmarkSingleRunbook-4

main (-) #906 (562f8e9) +/-
Number of iterations 18 18 0
Nanoseconds per iteration 66,976,424 ns/op 66,406,478 ns/op -569,946 ns/op
Bytes allocated per iteration 60,295,408 B/op 60,325,868 B/op 30,460 B/op
Allocs per iteration 205,871 allocs/op 205,898 allocs/op 27 allocs/op
Metadata | | [main](https://github.com/k1LoW/runn/tree/main) (-) | [#906](https://github.com/k1LoW/runn/pull/906) ([562f8e9](https://github.com/k1LoW/runn/commit/562f8e9e55177a6f52b8674821c77b88740554df)) | |------------|----------------------------------------------------:|------------------------------------------------------------------------------------------------------------------------------------------:| | **goos** | linux | linux | | **goarch** | amd64 | amd64 | | **pkg** | github.com/k1LoW/runn | github.com/k1LoW/runn | | **cpu** | AMD EPYC 7763 64-Core Processor | AMD EPYC 7763 64-Core Processor |

Reported by octocov

github-actions[bot] commented 4 months ago

Code Metrics Report

main (4d02f9c) #906 (fbb3409) +/-
Coverage 65.6% 65.6% 0.0%
Code to Test Ratio 1:0.5 1:0.5 -0.0
Details ``` diff | | main (4d02f9c) | #906 (fbb3409) | +/- | |--------------------|----------------|----------------|------| | Coverage | 65.6% | 65.6% | 0.0% | | Files | 71 | 71 | 0 | | Lines | 7685 | 7685 | 0 | | Covered | 5041 | 5041 | 0 | - | Code to Test Ratio | 1:0.5 | 1:0.5 | -0.0 | | Code | 22370 | 22370 | 0 | - | Test | 10279 | 10275 | -4 | ```

Reported by octocov

github-actions[bot] commented 4 months ago

Code Metrics Report

main (4d02f9c) #906 (562f8e9) +/-
Coverage 65.6% 65.6% 0.0%
Code to Test Ratio 1:0.5 1:0.5 -0.0
Details ``` diff | | main (4d02f9c) | #906 (562f8e9) | +/- | |--------------------|----------------|----------------|------| | Coverage | 65.6% | 65.6% | 0.0% | | Files | 71 | 71 | 0 | | Lines | 7685 | 7685 | 0 | | Covered | 5041 | 5041 | 0 | - | Code to Test Ratio | 1:0.5 | 1:0.5 | -0.0 | | Code | 22370 | 22370 | 0 | - | Test | 10279 | 10275 | -4 | ```

Reported by octocov

github-actions[bot] commented 4 months ago

BenchmarkManyRunbooks-4

main (-) #906 (d189ceb) +/-
Number of iterations 1 1 0
Nanoseconds per iteration 3,990,409,244 ns/op 4,101,953,583 ns/op 111,544,339 ns/op
Bytes allocated per iteration 1,792,926,072 B/op 1,792,921,464 B/op -4,608 B/op
Allocs per iteration 21,417,732 allocs/op 21,418,489 allocs/op 757 allocs/op
Metadata | | [main](https://github.com/k1LoW/runn/tree/main) (-) | [#906](https://github.com/k1LoW/runn/pull/906) ([d189ceb](https://github.com/k1LoW/runn/commit/d189ceb9505d87831848f352c9a1c0a07e58cb08)) | |------------|----------------------------------------------------:|------------------------------------------------------------------------------------------------------------------------------------------:| | **goos** | linux | linux | | **goarch** | amd64 | amd64 | | **pkg** | github.com/k1LoW/runn | github.com/k1LoW/runn | | **cpu** | AMD EPYC 7763 64-Core Processor | AMD EPYC 7763 64-Core Processor |

BenchmarkOpenAPI3-4

main (-) #906 (d189ceb) +/-
Number of iterations 1 1 0
Nanoseconds per iteration 5,118,233,159 ns/op 5,355,435,021 ns/op 237,201,862 ns/op
Bytes allocated per iteration 2,945,266,552 B/op 2,944,824,392 B/op -442,160 B/op
Allocs per iteration 29,265,044 allocs/op 29,261,927 allocs/op -3,117 allocs/op
Metadata | | [main](https://github.com/k1LoW/runn/tree/main) (-) | [#906](https://github.com/k1LoW/runn/pull/906) ([d189ceb](https://github.com/k1LoW/runn/commit/d189ceb9505d87831848f352c9a1c0a07e58cb08)) | |------------|----------------------------------------------------:|------------------------------------------------------------------------------------------------------------------------------------------:| | **goos** | linux | linux | | **goarch** | amd64 | amd64 | | **pkg** | github.com/k1LoW/runn | github.com/k1LoW/runn | | **cpu** | AMD EPYC 7763 64-Core Processor | AMD EPYC 7763 64-Core Processor |

BenchmarkSingleRunbook-4

main (-) #906 (d189ceb) +/-
Number of iterations 16 18 2
Nanoseconds per iteration 70,784,633 ns/op 70,842,880 ns/op 58,247 ns/op
Bytes allocated per iteration 60,342,536 B/op 60,312,016 B/op -30,520 B/op
Allocs per iteration 205,886 allocs/op 205,899 allocs/op 13 allocs/op
Metadata | | [main](https://github.com/k1LoW/runn/tree/main) (-) | [#906](https://github.com/k1LoW/runn/pull/906) ([d189ceb](https://github.com/k1LoW/runn/commit/d189ceb9505d87831848f352c9a1c0a07e58cb08)) | |------------|----------------------------------------------------:|------------------------------------------------------------------------------------------------------------------------------------------:| | **goos** | linux | linux | | **goarch** | amd64 | amd64 | | **pkg** | github.com/k1LoW/runn | github.com/k1LoW/runn | | **cpu** | AMD EPYC 7763 64-Core Processor | AMD EPYC 7763 64-Core Processor |

Reported by octocov

github-actions[bot] commented 4 months ago

Code Metrics Report

main (e81e116) #906 (d189ceb) +/-
Coverage 65.6% 65.6% 0.0%
Code to Test Ratio 1:0.5 1:0.5 -0.0
Details ``` diff | | main (e81e116) | #906 (d189ceb) | +/- | |--------------------|----------------|----------------|------| | Coverage | 65.6% | 65.6% | 0.0% | | Files | 71 | 71 | 0 | | Lines | 7685 | 7685 | 0 | | Covered | 5041 | 5041 | 0 | - | Code to Test Ratio | 1:0.5 | 1:0.5 | -0.0 | | Code | 22370 | 22370 | 0 | - | Test | 10279 | 10274 | -5 | ```

Reported by octocov

github-actions[bot] commented 4 months ago

BenchmarkManyRunbooks-4

main (-) #906 (b09308e) +/-
Number of iterations 1 1 0
Nanoseconds per iteration 3,720,463,283 ns/op 3,836,738,287 ns/op 116,275,004 ns/op
Bytes allocated per iteration 1,792,189,472 B/op 1,792,729,872 B/op 540,400 B/op
Allocs per iteration 21,406,244 allocs/op 21,406,751 allocs/op 507 allocs/op
Metadata | | [main](https://github.com/k1LoW/runn/tree/main) (-) | [#906](https://github.com/k1LoW/runn/pull/906) ([b09308e](https://github.com/k1LoW/runn/commit/b09308e7096c4e37887218c48c1db5137acab7f4)) | |------------|----------------------------------------------------:|------------------------------------------------------------------------------------------------------------------------------------------:| | **goos** | linux | linux | | **goarch** | amd64 | amd64 | | **pkg** | github.com/k1LoW/runn | github.com/k1LoW/runn | | **cpu** | AMD EPYC 7763 64-Core Processor | AMD EPYC 7763 64-Core Processor |

BenchmarkOpenAPI3-4

main (-) #906 (b09308e) +/-
Number of iterations 1 1 0
Nanoseconds per iteration 5,084,795,093 ns/op 5,034,029,823 ns/op -50,765,270 ns/op
Bytes allocated per iteration 2,944,897,448 B/op 2,944,958,488 B/op 61,040 B/op
Allocs per iteration 29,262,878 allocs/op 29,263,019 allocs/op 141 allocs/op
Metadata | | [main](https://github.com/k1LoW/runn/tree/main) (-) | [#906](https://github.com/k1LoW/runn/pull/906) ([b09308e](https://github.com/k1LoW/runn/commit/b09308e7096c4e37887218c48c1db5137acab7f4)) | |------------|----------------------------------------------------:|------------------------------------------------------------------------------------------------------------------------------------------:| | **goos** | linux | linux | | **goarch** | amd64 | amd64 | | **pkg** | github.com/k1LoW/runn | github.com/k1LoW/runn | | **cpu** | AMD EPYC 7763 64-Core Processor | AMD EPYC 7763 64-Core Processor |

BenchmarkSingleRunbook-4

main (-) #906 (b09308e) +/-
Number of iterations 18 18 0
Nanoseconds per iteration 65,349,594 ns/op 65,412,495 ns/op 62,901 ns/op
Bytes allocated per iteration 60,297,307 B/op 60,277,138 B/op -20,169 B/op
Allocs per iteration 205,776 allocs/op 205,750 allocs/op -26 allocs/op
Metadata | | [main](https://github.com/k1LoW/runn/tree/main) (-) | [#906](https://github.com/k1LoW/runn/pull/906) ([b09308e](https://github.com/k1LoW/runn/commit/b09308e7096c4e37887218c48c1db5137acab7f4)) | |------------|----------------------------------------------------:|------------------------------------------------------------------------------------------------------------------------------------------:| | **goos** | linux | linux | | **goarch** | amd64 | amd64 | | **pkg** | github.com/k1LoW/runn | github.com/k1LoW/runn | | **cpu** | AMD EPYC 7763 64-Core Processor | AMD EPYC 7763 64-Core Processor |

Reported by octocov

github-actions[bot] commented 4 months ago

Code Metrics Report

main (e81e116) #906 (b09308e) +/-
Coverage 65.6% 65.6% 0.0%
Code to Test Ratio 1:0.5 1:0.5 +0.0
Details ``` diff | | main (e81e116) | #906 (b09308e) | +/- | |--------------------|----------------|----------------|------| | Coverage | 65.6% | 65.6% | 0.0% | | Files | 71 | 71 | 0 | | Lines | 7685 | 7685 | 0 | | Covered | 5041 | 5041 | 0 | + | Code to Test Ratio | 1:0.5 | 1:0.5 | +0.0 | | Code | 22370 | 22370 | 0 | + | Test | 10279 | 10452 | +173 | ```

Reported by octocov

github-actions[bot] commented 4 months ago

BenchmarkManyRunbooks-4

main (-) #906 (93fc300) +/-
Number of iterations 1 1 0
Nanoseconds per iteration 3,856,903,295 ns/op 4,020,677,693 ns/op 163,774,398 ns/op
Bytes allocated per iteration 1,792,193,776 B/op 1,791,858,856 B/op -334,920 B/op
Allocs per iteration 21,406,252 allocs/op 21,405,869 allocs/op -383 allocs/op
Metadata | | [main](https://github.com/k1LoW/runn/tree/main) (-) | [#906](https://github.com/k1LoW/runn/pull/906) ([93fc300](https://github.com/k1LoW/runn/commit/93fc300b41afa3032dc7d65892b787c6e13ede22)) | |------------|----------------------------------------------------:|------------------------------------------------------------------------------------------------------------------------------------------:| | **goos** | linux | linux | | **goarch** | amd64 | amd64 | | **pkg** | github.com/k1LoW/runn | github.com/k1LoW/runn | | **cpu** | AMD EPYC 7763 64-Core Processor | AMD EPYC 7763 64-Core Processor |

BenchmarkOpenAPI3-4

main (-) #906 (93fc300) +/-
Number of iterations 1 1 0
Nanoseconds per iteration 5,212,513,288 ns/op 5,053,755,701 ns/op -158,757,587 ns/op
Bytes allocated per iteration 2,944,857,912 B/op 2,944,893,032 B/op 35,120 B/op
Allocs per iteration 29,271,428 allocs/op 29,263,107 allocs/op -8,321 allocs/op
Metadata | | [main](https://github.com/k1LoW/runn/tree/main) (-) | [#906](https://github.com/k1LoW/runn/pull/906) ([93fc300](https://github.com/k1LoW/runn/commit/93fc300b41afa3032dc7d65892b787c6e13ede22)) | |------------|----------------------------------------------------:|------------------------------------------------------------------------------------------------------------------------------------------:| | **goos** | linux | linux | | **goarch** | amd64 | amd64 | | **pkg** | github.com/k1LoW/runn | github.com/k1LoW/runn | | **cpu** | AMD EPYC 7763 64-Core Processor | AMD EPYC 7763 64-Core Processor |

BenchmarkSingleRunbook-4

main (-) #906 (93fc300) +/-
Number of iterations 16 16 0
Nanoseconds per iteration 68,656,488 ns/op 67,554,702 ns/op -1,101,786 ns/op
Bytes allocated per iteration 60,271,295 B/op 60,280,316 B/op 9,021 B/op
Allocs per iteration 205,744 allocs/op 205,768 allocs/op 24 allocs/op
Metadata | | [main](https://github.com/k1LoW/runn/tree/main) (-) | [#906](https://github.com/k1LoW/runn/pull/906) ([93fc300](https://github.com/k1LoW/runn/commit/93fc300b41afa3032dc7d65892b787c6e13ede22)) | |------------|----------------------------------------------------:|------------------------------------------------------------------------------------------------------------------------------------------:| | **goos** | linux | linux | | **goarch** | amd64 | amd64 | | **pkg** | github.com/k1LoW/runn | github.com/k1LoW/runn | | **cpu** | AMD EPYC 7763 64-Core Processor | AMD EPYC 7763 64-Core Processor |

Reported by octocov

github-actions[bot] commented 4 months ago

Code Metrics Report

main (e81e116) #906 (93fc300) +/-
Coverage 65.6% 65.6% 0.0%
Code to Test Ratio 1:0.5 1:0.5 +0.0
Details ``` diff | | main (e81e116) | #906 (93fc300) | +/- | |--------------------|----------------|----------------|------| | Coverage | 65.6% | 65.6% | 0.0% | | Files | 71 | 71 | 0 | | Lines | 7685 | 7685 | 0 | | Covered | 5041 | 5041 | 0 | + | Code to Test Ratio | 1:0.5 | 1:0.5 | +0.0 | | Code | 22370 | 22370 | 0 | + | Test | 10279 | 10468 | +189 | ```

Code coverage of files in pull request scope (64.4% → 64.4%)

Files Coverage +/-
coverage.go 79.5% 0.0%
http_validator.go 81.5% 0.0%
runner_option.go 37.5% 0.0%

Reported by octocov

github-actions[bot] commented 4 months ago

BenchmarkManyRunbooks-4

main (-) #906 (615a192) +/-
Number of iterations 1 1 0
Nanoseconds per iteration 3,828,637,136 ns/op 4,030,704,198 ns/op 202,067,062 ns/op
Bytes allocated per iteration 1,792,101,208 B/op 1,792,380,200 B/op 278,992 B/op
Allocs per iteration 21,406,167 allocs/op 21,406,490 allocs/op 323 allocs/op
Metadata | | [main](https://github.com/k1LoW/runn/tree/main) (-) | [#906](https://github.com/k1LoW/runn/pull/906) ([615a192](https://github.com/k1LoW/runn/commit/615a192c14cd4620265202f409ce133d99449b0c)) | |------------|----------------------------------------------------:|------------------------------------------------------------------------------------------------------------------------------------------:| | **goos** | linux | linux | | **goarch** | amd64 | amd64 | | **pkg** | github.com/k1LoW/runn | github.com/k1LoW/runn | | **cpu** | AMD EPYC 7763 64-Core Processor | AMD EPYC 7763 64-Core Processor |

BenchmarkOpenAPI3-4

main (-) #906 (615a192) +/-
Number of iterations 1 1 0
Nanoseconds per iteration 5,224,460,322 ns/op 5,139,860,338 ns/op -84,599,984 ns/op
Bytes allocated per iteration 2,945,095,240 B/op 2,945,167,624 B/op 72,384 B/op
Allocs per iteration 29,267,357 allocs/op 29,270,848 allocs/op 3,491 allocs/op
Metadata | | [main](https://github.com/k1LoW/runn/tree/main) (-) | [#906](https://github.com/k1LoW/runn/pull/906) ([615a192](https://github.com/k1LoW/runn/commit/615a192c14cd4620265202f409ce133d99449b0c)) | |------------|----------------------------------------------------:|------------------------------------------------------------------------------------------------------------------------------------------:| | **goos** | linux | linux | | **goarch** | amd64 | amd64 | | **pkg** | github.com/k1LoW/runn | github.com/k1LoW/runn | | **cpu** | AMD EPYC 7763 64-Core Processor | AMD EPYC 7763 64-Core Processor |

BenchmarkSingleRunbook-4

main (-) #906 (615a192) +/-
Number of iterations 16 16 0
Nanoseconds per iteration 66,868,862 ns/op 65,286,193 ns/op -1,582,669 ns/op
Bytes allocated per iteration 60,304,917 B/op 60,265,084 B/op -39,833 B/op
Allocs per iteration 205,787 allocs/op 205,743 allocs/op -44 allocs/op
Metadata | | [main](https://github.com/k1LoW/runn/tree/main) (-) | [#906](https://github.com/k1LoW/runn/pull/906) ([615a192](https://github.com/k1LoW/runn/commit/615a192c14cd4620265202f409ce133d99449b0c)) | |------------|----------------------------------------------------:|------------------------------------------------------------------------------------------------------------------------------------------:| | **goos** | linux | linux | | **goarch** | amd64 | amd64 | | **pkg** | github.com/k1LoW/runn | github.com/k1LoW/runn | | **cpu** | AMD EPYC 7763 64-Core Processor | AMD EPYC 7763 64-Core Processor |

Reported by octocov

github-actions[bot] commented 4 months ago

Code Metrics Report

main (e81e116) #906 (615a192) +/-
Coverage 65.6% 65.6% 0.0%
Code to Test Ratio 1:0.5 1:0.5 +0.0
Details ``` diff | | main (e81e116) | #906 (615a192) | +/- | |--------------------|----------------|----------------|------| | Coverage | 65.6% | 65.6% | 0.0% | | Files | 71 | 71 | 0 | | Lines | 7685 | 7685 | 0 | | Covered | 5041 | 5041 | 0 | + | Code to Test Ratio | 1:0.5 | 1:0.5 | +0.0 | | Code | 22370 | 22370 | 0 | + | Test | 10279 | 10483 | +204 | ```

Code coverage of files in pull request scope (64.4% → 64.4%)

Files Coverage +/-
coverage.go 79.5% 0.0%
http_validator.go 81.5% 0.0%
runner_option.go 37.5% 0.0%

Reported by octocov