Randomizing Tests in the CI is sometimes showing bugs from what was first tested, as it was not randomized. This was due to some tests affecting the other tests, it was seen when CI was being run. To mitigate this changed are introduced, and one test does not affect the other.
Related Tickets & Documents
Related PR #17
Checklist before requesting a review
[x] I have performed a self-review of my code.
[ ] If it is a core feature, I have added thorough tests.
Testing
Please describe the System Under Test.
Please provide detailed steps to perform tests related to this code change.
How were the fix/results from this change verified? Please provide relevant screenshots or results.
Results after randomizing
ginkgo -r --randomize-all --randomize-suites --fail-on-pending --cover --trace --v --coverprofile=coverage.out ./..
Running Suite: Prometheus Suite - /home/sboyapal/Documents/forkedCommons/go-commons/prometheus
==============================================================================================
Random Seed: 1688731387 - will randomize all specs
Will run 15 of 15 specs
Tests for Prometheus Test for QueryRangeAggregatedTS Test7 for P50 etc.
/home/sboyapal/Documents/forkedCommons/go-commons/prometheus/prometheus_test.go:174
• [0.000 seconds]
Tests for Prometheus Test for QueryRangeAggregatedTS Test2 for Avg
/home/sboyapal/Documents/forkedCommons/go-commons/prometheus/prometheus_test.go:138
• [0.000 seconds]
Tests for Prometheus Tests for RoundTrip() Test1 for default behaviour
/home/sboyapal/Documents/forkedCommons/go-commons/prometheus/prometheus_test.go:25
• [0.157 seconds]
Tests for Prometheus Tests for NewClient() Test1 empty parameters
/home/sboyapal/Documents/forkedCommons/go-commons/prometheus/prometheus_test.go:50
• [0.000 seconds]
Tests for Prometheus Test for QueryRangeAggregatedTS Test3 error when v not in matrix
/home/sboyapal/Documents/forkedCommons/go-commons/prometheus/prometheus_test.go:145
• [0.000 seconds]
Tests for Prometheus Tests for Query() Test1 empty url
/home/sboyapal/Documents/forkedCommons/go-commons/prometheus/prometheus_test.go:76
• [0.000 seconds]
Tests for Prometheus Test for QueryRangeAggregatedTS Test1 queryRange error
/home/sboyapal/Documents/forkedCommons/go-commons/prometheus/prometheus_test.go:130
• [0.000 seconds]
Tests for Prometheus Test for QueryRangeAggregatedTS Test4 for Min
/home/sboyapal/Documents/forkedCommons/go-commons/prometheus/prometheus_test.go:153
• [0.000 seconds]
Tests for Prometheus Tests for NewClient() Test2 passing not valid url
/home/sboyapal/Documents/forkedCommons/go-commons/prometheus/prometheus_test.go:57
• [0.000 seconds]
Tests for Prometheus Test for QueryRangeAggregatedTS Test6 no standard aggregator
/home/sboyapal/Documents/forkedCommons/go-commons/prometheus/prometheus_test.go:181
• [0.000 seconds]
Tests for Prometheus Test for QueryRangeAggregatedTS Test6 for Stdev
/home/sboyapal/Documents/forkedCommons/go-commons/prometheus/prometheus_test.go:167
• [0.000 seconds]
Tests for Prometheus Tests for Query() Test2 mock error to nil
/home/sboyapal/Documents/forkedCommons/go-commons/prometheus/prometheus_test.go:83
• [0.000 seconds]
Tests for Prometheus Test for QueryRangeAggregatedTS Test5 for Max
/home/sboyapal/Documents/forkedCommons/go-commons/prometheus/prometheus_test.go:160
• [0.000 seconds]
Tests for Prometheus Tests for verifyConnection() Test1 mock to no nil
/home/sboyapal/Documents/forkedCommons/go-commons/prometheus/prometheus_test.go:197
• [0.000 seconds]
Tests for Prometheus Tests for QueryRange() Test1 empty url
/home/sboyapal/Documents/forkedCommons/go-commons/prometheus/prometheus_test.go:104
• [0.000 seconds]
Ran 15 of 15 Specs in 0.159 seconds
SUCCESS! -- 15 Passed | 0 Failed | 0 Pending | 0 Skipped
PASS
github.com/cloud-bulldozer/go-commons/prometheus coverage: 98.0% of statements
Running Suite: Indexers Suite - /home/sboyapal/Documents/forkedCommons/go-commons/indexers
Random Seed: 1688731387 - will randomize all specs
Tests for local.go Default behaviour of local.go, Index() No err is returned
/home/sboyapal/Documents/forkedCommons/go-commons/indexers/local_test.go:76
• [0.000 seconds]
Factory.go Unit Tests: NewIndexer() Default behaviour of NewIndexer() returns indexer and err status bad gateway
/home/sboyapal/Documents/forkedCommons/go-commons/indexers/factory_test.go:41
• [0.001 seconds]
Tests for opensearch.go Tests for new() Returns err no index name
/home/sboyapal/Documents/forkedCommons/go-commons/indexers/opensearch_test.go:63
• [0.000 seconds]
Tests for elastic.go Tests for new() when no url is passed
/home/sboyapal/Documents/forkedCommons/go-commons/indexers/elastic_test.go:47
• [0.009 seconds]
Factory.go Unit Tests: NewIndexer() Default behaviour of NewIndexer() returns indexer and err unknown indexer
/home/sboyapal/Documents/forkedCommons/go-commons/indexers/factory_test.go:52
• [0.000 seconds]
Tests for elastic.go Tests for new() Returns err no index name
/home/sboyapal/Documents/forkedCommons/go-commons/indexers/elastic_test.go:65
• [0.000 seconds]
Tests for opensearch.go Tests for new() Returns error
/home/sboyapal/Documents/forkedCommons/go-commons/indexers/opensearch_test.go:36
• [0.000 seconds]
Factory.go Unit Tests: NewIndexer() Default behaviour of NewIndexer() returns indexer and nil
/home/sboyapal/Documents/forkedCommons/go-commons/indexers/factory_test.go:84
• [0.001 seconds]
Tests for elastic.go Tests for new() Returns err not passing a valid URL in env variable
/home/sboyapal/Documents/forkedCommons/go-commons/indexers/elastic_test.go:56
• [0.000 seconds]
Tests for opensearch.go Tests for new() Returns err not a valid URL in env variable
/home/sboyapal/Documents/forkedCommons/go-commons/indexers/opensearch_test.go:54
• [0.000 seconds]
Tests for opensearch.go Tests for Index() No err returned
/home/sboyapal/Documents/forkedCommons/go-commons/indexers/opensearch_test.go:101
• [0.041 seconds]
Tests for elastic.go Tests for Index() No err returned
/home/sboyapal/Documents/forkedCommons/go-commons/indexers/elastic_test.go:103
• [0.041 seconds]
Tests for local.go Default behavior of local.go, new() returns err no metrics directory
/home/sboyapal/Documents/forkedCommons/go-commons/indexers/local_test.go:31
• [0.000 seconds]
Tests for local.go Default behaviour of local.go, Index() Err is returned metricsdirectory has fault
/home/sboyapal/Documents/forkedCommons/go-commons/indexers/local_test.go:86
• [0.000 seconds]
Tests for elastic.go Tests for Index() err returned docs not processed
/home/sboyapal/Documents/forkedCommons/go-commons/indexers/elastic_test.go:108
• [0.003 seconds]
Tests for local.go Default behaviour of local.go, Index() No err is returned
/home/sboyapal/Documents/forkedCommons/go-commons/indexers/local_test.go:81
• [0.000 seconds]
Factory.go Unit Tests: NewIndexer() Default behaviour of NewIndexer() returns indexer and nil
/home/sboyapal/Documents/forkedCommons/go-commons/indexers/factory_test.go:34
• [0.001 seconds]
Tests for local.go Default behaviour of local.go, Index() Err is returned by documents not processed
/home/sboyapal/Documents/forkedCommons/go-commons/indexers/local_test.go:92
• [0.000 seconds]
Tests for opensearch.go Tests for new() when no url is passed
/home/sboyapal/Documents/forkedCommons/go-commons/indexers/opensearch_test.go:46
• [0.006 seconds]
Tests for elastic.go Tests for new() Returns error status bad request
/home/sboyapal/Documents/forkedCommons/go-commons/indexers/elastic_test.go:37
• [0.000 seconds]
Tests for local.go Default behavior of local.go, new() returns nil as error
/home/sboyapal/Documents/forkedCommons/go-commons/indexers/local_test.go:36
• [0.000 seconds]
Tests for opensearch.go Tests for Index() err returned docs not processed
/home/sboyapal/Documents/forkedCommons/go-commons/indexers/opensearch_test.go:106
• [0.003 seconds]
Ran 22 of 22 Specs in 0.109 seconds
SUCCESS! -- 22 Passed | 0 Failed | 0 Pending | 0 Skipped
PASS
github.com/cloud-bulldozer/go-commons/indexers coverage: 81.8% of statements
Running Suite: Comparison Suite - /home/sboyapal/Documents/forkedCommons/go-commons/comparison
Random Seed: 1688731387 - will randomize all specs
Will run 11 of 11 specs
Tests for elastic.go Tests on Compare() Test5 no error with sum stat
/home/sboyapal/Documents/forkedCommons/go-commons/comparison/elastic_test.go:186
• [0.301 seconds]
Tests for elastic.go Tests on Compare() Test2 negative value
/home/sboyapal/Documents/forkedCommons/go-commons/comparison/elastic_test.go:159
• [0.001 seconds]
Tests for elastic.go Tests for NewComparator Test 1: Returns the expected comparator
/home/sboyapal/Documents/forkedCommons/go-commons/comparison/elastic_test.go:40
• [0.000 seconds]
Tests for elastic.go Tests for queryStringStats() Test1 error 404
/home/sboyapal/Documents/forkedCommons/go-commons/comparison/elastic_test.go:66
• [0.000 seconds]
Tests for elastic.go Tests on Compare() Test3 negative tolerance
/home/sboyapal/Documents/forkedCommons/go-commons/comparison/elastic_test.go:168
• [0.000 seconds]
Tests for elastic.go Tests for queryStringStats() Test3 not a valid link
/home/sboyapal/Documents/forkedCommons/go-commons/comparison/elastic_test.go:93
• [0.244 seconds]
Tests for elastic.go Tests on Compare() Test1 no error
/home/sboyapal/Documents/forkedCommons/go-commons/comparison/elastic_test.go:150
• [0.000 seconds]
Tests for elastic.go Tests on Compare() Test4 negative tolerance
/home/sboyapal/Documents/forkedCommons/go-commons/comparison/elastic_test.go:177
• [0.000 seconds]
Tests for elastic.go Tests for queryStringStats() Test2 no error
/home/sboyapal/Documents/forkedCommons/go-commons/comparison/elastic_test.go:83
• [0.000 seconds]
Tests for elastic.go Tests for queryStringStats() Test5 non parsable json
/home/sboyapal/Documents/forkedCommons/go-commons/comparison/elastic_test.go:119
• [0.000 seconds]
Tests for elastic.go Tests for queryStringStats() Test4 non parsable json
/home/sboyapal/Documents/forkedCommons/go-commons/comparison/elastic_test.go:101
• [0.000 seconds]
Ran 11 of 11 Specs in 0.547 seconds
SUCCESS! -- 11 Passed | 0 Failed | 0 Pending | 0 Skipped
PASS
github.com/cloud-bulldozer/go-commons/comparison coverage: 100.0% of statements
composite coverage: 88.7% of statements
Ginkgo ran 3 suites in 1.592207584s
Test Suite Passed
Bug Fix comparison elastic.go Unit Test
Description
Randomizing Tests in the CI is sometimes showing bugs from what was first tested, as it was not randomized. This was due to some tests affecting the other tests, it was seen when CI was being run. To mitigate this changed are introduced, and one test does not affect the other.
Related Tickets & Documents
Checklist before requesting a review
Testing
Will run 15 of 15 specs
Tests for Prometheus Test for QueryRangeAggregatedTS Test7 for P50 etc. /home/sboyapal/Documents/forkedCommons/go-commons/prometheus/prometheus_test.go:174 • [0.000 seconds]
Tests for Prometheus Test for QueryRangeAggregatedTS Test2 for Avg /home/sboyapal/Documents/forkedCommons/go-commons/prometheus/prometheus_test.go:138 • [0.000 seconds]
Tests for Prometheus Tests for RoundTrip() Test1 for default behaviour /home/sboyapal/Documents/forkedCommons/go-commons/prometheus/prometheus_test.go:25 • [0.157 seconds]
Tests for Prometheus Tests for NewClient() Test1 empty parameters /home/sboyapal/Documents/forkedCommons/go-commons/prometheus/prometheus_test.go:50 • [0.000 seconds]
Tests for Prometheus Test for QueryRangeAggregatedTS Test3 error when v not in matrix /home/sboyapal/Documents/forkedCommons/go-commons/prometheus/prometheus_test.go:145 • [0.000 seconds]
Tests for Prometheus Tests for Query() Test1 empty url /home/sboyapal/Documents/forkedCommons/go-commons/prometheus/prometheus_test.go:76 • [0.000 seconds]
Tests for Prometheus Test for QueryRangeAggregatedTS Test1 queryRange error /home/sboyapal/Documents/forkedCommons/go-commons/prometheus/prometheus_test.go:130 • [0.000 seconds]
Tests for Prometheus Test for QueryRangeAggregatedTS Test4 for Min /home/sboyapal/Documents/forkedCommons/go-commons/prometheus/prometheus_test.go:153 • [0.000 seconds]
Tests for Prometheus Tests for NewClient() Test2 passing not valid url /home/sboyapal/Documents/forkedCommons/go-commons/prometheus/prometheus_test.go:57 • [0.000 seconds]
Tests for Prometheus Test for QueryRangeAggregatedTS Test6 no standard aggregator /home/sboyapal/Documents/forkedCommons/go-commons/prometheus/prometheus_test.go:181 • [0.000 seconds]
Tests for Prometheus Test for QueryRangeAggregatedTS Test6 for Stdev /home/sboyapal/Documents/forkedCommons/go-commons/prometheus/prometheus_test.go:167 • [0.000 seconds]
Tests for Prometheus Tests for Query() Test2 mock error to nil /home/sboyapal/Documents/forkedCommons/go-commons/prometheus/prometheus_test.go:83 • [0.000 seconds]
Tests for Prometheus Test for QueryRangeAggregatedTS Test5 for Max /home/sboyapal/Documents/forkedCommons/go-commons/prometheus/prometheus_test.go:160 • [0.000 seconds]
Tests for Prometheus Tests for verifyConnection() Test1 mock to no nil /home/sboyapal/Documents/forkedCommons/go-commons/prometheus/prometheus_test.go:197 • [0.000 seconds]
Tests for Prometheus Tests for QueryRange() Test1 empty url /home/sboyapal/Documents/forkedCommons/go-commons/prometheus/prometheus_test.go:104 • [0.000 seconds]
Ran 15 of 15 Specs in 0.159 seconds SUCCESS! -- 15 Passed | 0 Failed | 0 Pending | 0 Skipped PASS github.com/cloud-bulldozer/go-commons/prometheus coverage: 98.0% of statements Running Suite: Indexers Suite - /home/sboyapal/Documents/forkedCommons/go-commons/indexers
Random Seed: 1688731387 - will randomize all specs
Will run 22 of 22 specs
[BeforeSuite] /home/sboyapal/Documents/forkedCommons/go-commons/indexers/test_assets_test.go:10 [BeforeSuite] PASSED [0.000 seconds]
Tests for local.go Default behaviour of local.go, Index() No err is returned /home/sboyapal/Documents/forkedCommons/go-commons/indexers/local_test.go:76 • [0.000 seconds]
Factory.go Unit Tests: NewIndexer() Default behaviour of NewIndexer() returns indexer and err status bad gateway /home/sboyapal/Documents/forkedCommons/go-commons/indexers/factory_test.go:41 • [0.001 seconds]
Tests for opensearch.go Tests for new() Returns err no index name /home/sboyapal/Documents/forkedCommons/go-commons/indexers/opensearch_test.go:63 • [0.000 seconds]
Tests for elastic.go Tests for new() when no url is passed /home/sboyapal/Documents/forkedCommons/go-commons/indexers/elastic_test.go:47 • [0.009 seconds]
Factory.go Unit Tests: NewIndexer() Default behaviour of NewIndexer() returns indexer and err unknown indexer /home/sboyapal/Documents/forkedCommons/go-commons/indexers/factory_test.go:52 • [0.000 seconds]
Tests for elastic.go Tests for new() Returns err no index name /home/sboyapal/Documents/forkedCommons/go-commons/indexers/elastic_test.go:65 • [0.000 seconds]
Tests for opensearch.go Tests for new() Returns error /home/sboyapal/Documents/forkedCommons/go-commons/indexers/opensearch_test.go:36 • [0.000 seconds]
Factory.go Unit Tests: NewIndexer() Default behaviour of NewIndexer() returns indexer and nil /home/sboyapal/Documents/forkedCommons/go-commons/indexers/factory_test.go:84 • [0.001 seconds]
Tests for elastic.go Tests for new() Returns err not passing a valid URL in env variable /home/sboyapal/Documents/forkedCommons/go-commons/indexers/elastic_test.go:56 • [0.000 seconds]
Tests for opensearch.go Tests for new() Returns err not a valid URL in env variable /home/sboyapal/Documents/forkedCommons/go-commons/indexers/opensearch_test.go:54 • [0.000 seconds]
Tests for opensearch.go Tests for Index() No err returned /home/sboyapal/Documents/forkedCommons/go-commons/indexers/opensearch_test.go:101 • [0.041 seconds]
Tests for elastic.go Tests for Index() No err returned /home/sboyapal/Documents/forkedCommons/go-commons/indexers/elastic_test.go:103 • [0.041 seconds]
Tests for local.go Default behavior of local.go, new() returns err no metrics directory /home/sboyapal/Documents/forkedCommons/go-commons/indexers/local_test.go:31 • [0.000 seconds]
Tests for local.go Default behaviour of local.go, Index() Err is returned metricsdirectory has fault /home/sboyapal/Documents/forkedCommons/go-commons/indexers/local_test.go:86 • [0.000 seconds]
Tests for elastic.go Tests for Index() err returned docs not processed /home/sboyapal/Documents/forkedCommons/go-commons/indexers/elastic_test.go:108 • [0.003 seconds]
Tests for local.go Default behaviour of local.go, Index() No err is returned /home/sboyapal/Documents/forkedCommons/go-commons/indexers/local_test.go:81 • [0.000 seconds]
Factory.go Unit Tests: NewIndexer() Default behaviour of NewIndexer() returns indexer and nil /home/sboyapal/Documents/forkedCommons/go-commons/indexers/factory_test.go:34 • [0.001 seconds]
Tests for local.go Default behaviour of local.go, Index() Err is returned by documents not processed /home/sboyapal/Documents/forkedCommons/go-commons/indexers/local_test.go:92 • [0.000 seconds]
Tests for opensearch.go Tests for new() when no url is passed /home/sboyapal/Documents/forkedCommons/go-commons/indexers/opensearch_test.go:46 • [0.006 seconds]
Tests for elastic.go Tests for new() Returns error status bad request /home/sboyapal/Documents/forkedCommons/go-commons/indexers/elastic_test.go:37 • [0.000 seconds]
Tests for local.go Default behavior of local.go, new() returns nil as error /home/sboyapal/Documents/forkedCommons/go-commons/indexers/local_test.go:36 • [0.000 seconds]
Tests for opensearch.go Tests for Index() err returned docs not processed /home/sboyapal/Documents/forkedCommons/go-commons/indexers/opensearch_test.go:106 • [0.003 seconds]
Ran 22 of 22 Specs in 0.109 seconds SUCCESS! -- 22 Passed | 0 Failed | 0 Pending | 0 Skipped PASS github.com/cloud-bulldozer/go-commons/indexers coverage: 81.8% of statements Running Suite: Comparison Suite - /home/sboyapal/Documents/forkedCommons/go-commons/comparison
Random Seed: 1688731387 - will randomize all specs
Will run 11 of 11 specs
Tests for elastic.go Tests on Compare() Test5 no error with sum stat /home/sboyapal/Documents/forkedCommons/go-commons/comparison/elastic_test.go:186 • [0.301 seconds]
Tests for elastic.go Tests on Compare() Test2 negative value /home/sboyapal/Documents/forkedCommons/go-commons/comparison/elastic_test.go:159 • [0.001 seconds]
Tests for elastic.go Tests for NewComparator Test 1: Returns the expected comparator /home/sboyapal/Documents/forkedCommons/go-commons/comparison/elastic_test.go:40 • [0.000 seconds]
Tests for elastic.go Tests for queryStringStats() Test1 error 404 /home/sboyapal/Documents/forkedCommons/go-commons/comparison/elastic_test.go:66 • [0.000 seconds]
Tests for elastic.go Tests on Compare() Test3 negative tolerance /home/sboyapal/Documents/forkedCommons/go-commons/comparison/elastic_test.go:168 • [0.000 seconds]
Tests for elastic.go Tests for queryStringStats() Test3 not a valid link /home/sboyapal/Documents/forkedCommons/go-commons/comparison/elastic_test.go:93 • [0.244 seconds]
Tests for elastic.go Tests on Compare() Test1 no error /home/sboyapal/Documents/forkedCommons/go-commons/comparison/elastic_test.go:150 • [0.000 seconds]
Tests for elastic.go Tests on Compare() Test4 negative tolerance /home/sboyapal/Documents/forkedCommons/go-commons/comparison/elastic_test.go:177 • [0.000 seconds]
Tests for elastic.go Tests for queryStringStats() Test2 no error /home/sboyapal/Documents/forkedCommons/go-commons/comparison/elastic_test.go:83 • [0.000 seconds]
Tests for elastic.go Tests for queryStringStats() Test5 non parsable json /home/sboyapal/Documents/forkedCommons/go-commons/comparison/elastic_test.go:119 • [0.000 seconds]
Tests for elastic.go Tests for queryStringStats() Test4 non parsable json /home/sboyapal/Documents/forkedCommons/go-commons/comparison/elastic_test.go:101 • [0.000 seconds]
Ran 11 of 11 Specs in 0.547 seconds SUCCESS! -- 11 Passed | 0 Failed | 0 Pending | 0 Skipped PASS github.com/cloud-bulldozer/go-commons/comparison coverage: 100.0% of statements composite coverage: 88.7% of statements
Ginkgo ran 3 suites in 1.592207584s Test Suite Passed