cnti-testcatalog / testsuite

📞📱☎️📡🌐 Cloud Native Telecom Initiative (CNTI) Test Catalog is a tool to check for and provide feedback on the use of K8s + cloud native best practices in networking applications and platforms
https://wiki.lfnetworking.org/display/LN/Test+Catalog
Apache License 2.0
169 stars 70 forks source link

[BUG] Scalability spec test uses a tag in a non-uniform way. #1960

Closed svteb closed 2 months ago

svteb commented 2 months ago

Describe the bug

The header of scalability spec test has missplaced quotation marks around tags: -> tags: "[scalability]"

it "'scalability' should run all of the scalability tests", tags: "[scalability]"  do
    result = ShellCmd.run("./cnf-testsuite samples_cleanup")
    ...
    (/PASSED: Replicas decreased to 1/ =~ result[:output]).should_not be_nil
  end
end

All the other tags (except for the "WIP" tag) follow the format tags: ["tag-name"]:

it "Helm_values should be used during the installation of a cnf", tags: ["cnf-config"]  do
  ...
it "'no_local_volume_configuration' should pass if local storage configuration is not found", tags: ["no_local_volume_configuration"]  do
  ...
it "'rolling_version_change' should pass if using local registry and a port", tags: ["private_registry_version"]  do
  ...

Additional context Changing this should likely not break anything. Found in ~/testsuite/spec/workload/scalability_spec.

HashNuke commented 2 months ago

Yes. The value for tags should be an array of strings. This issue is valid. It should have been

tags: ["scalability"]

barmull commented 2 months ago

The tag should be fixed, but it might break github actions, because scalability test was being skipped.