Backblaze / boardwalk

A linear remote execution workflow engine built on top of Ansible
Other
18 stars 1 forks source link

[SVRENG-609] boardwalk (bug): correct job class initialization #86

Closed asullivan-blze closed 2 weeks ago

asullivan-blze commented 2 weeks ago

What and why?

Boardwalk release 0.8.22 introduced a bug where options supplied to Jobs would cause a TypeError, due to the options parameter not being included in the declaration of the init method.

This has been fixed, and to guard against similar mishaps, a test suite to exercise these regressions was added into the overall test suite, with focus on the options, as well as testing preconditions.

How was this tested?

Creation of the test suite located in test/server-client/pylib/regression_bz_svreng_609.py prior to directly working on the main issue, then addition of two additional test workflows to validate that preconditions continue to work as expected.

`make test` result
```bash (boardwalk-py3.12) asullivan@MBP-NT9RPG2XV7 boardwalk % make test poetry install Installing dependencies from lock file No dependencies to install or update Installing the current project: boardwalk (0.8.23) poetry run pytest --verbose =============================================================================================================== test session starts =============================================================================================================== platform darwin -- Python 3.12.1, pytest-8.3.3, pluggy-1.5.0 -- /Users/asullivan/Library/Caches/pypoetry/virtualenvs/boardwalk-mWVoODTS-py3.12/bin/python cachedir: .pytest_cache rootdir: /Users/asullivan/backblaze/github_repos/boardwalk configfile: pyproject.toml plugins: anyio-4.6.2.post1 collected 34 items test/boardwalk/test_boardwalk_cli.py::test_version PASSED [ 2%] test/boardwalk/test_manifest.py::test_using_not_differentiated_Job_class_warns_about_deprecation PASSED [ 5%] test/boardwalk/test_utils.py::test_strtobool[y-True] PASSED [ 8%] test/boardwalk/test_utils.py::test_strtobool[Y-True] PASSED [ 11%] test/boardwalk/test_utils.py::test_strtobool[yes-True] PASSED [ 14%] test/boardwalk/test_utils.py::test_strtobool[True-True0] PASSED [ 17%] test/boardwalk/test_utils.py::test_strtobool[t-True] PASSED [ 20%] test/boardwalk/test_utils.py::test_strtobool[true-True] PASSED [ 23%] test/boardwalk/test_utils.py::test_strtobool[True-True1] PASSED [ 26%] test/boardwalk/test_utils.py::test_strtobool[On-True] PASSED [ 29%] test/boardwalk/test_utils.py::test_strtobool[on-True] PASSED [ 32%] test/boardwalk/test_utils.py::test_strtobool[1-True] PASSED [ 35%] test/boardwalk/test_utils.py::test_strtobool[n-False] PASSED [ 38%] test/boardwalk/test_utils.py::test_strtobool[no-False] PASSED [ 41%] test/boardwalk/test_utils.py::test_strtobool[f-False] PASSED [ 44%] test/boardwalk/test_utils.py::test_strtobool[false-False] PASSED [ 47%] test/boardwalk/test_utils.py::test_strtobool[off-False] PASSED [ 50%] test/boardwalk/test_utils.py::test_strtobool[0-False] PASSED [ 52%] test/boardwalk/test_utils.py::test_strtobool[Off-False] PASSED [ 55%] test/boardwalk/test_utils.py::test_strtobool[No-False] PASSED [ 58%] test/boardwalk/test_utils.py::test_strtobool[N-False] PASSED [ 61%] test/boardwalkd/test_boardwalkd_cli.py::test_version PASSED [ 64%] test/boardwalkd/test_boardwalkd_cli.py::test_incomplete_serve_command[Missing required parameters to serve boardwalkd: --url, --host-header-pattern, --port/--tls-port-options0-2] PASSED [ 67%] test/boardwalkd/test_boardwalkd_cli.py::test_incomplete_serve_command[Missing required parameters to serve boardwalkd: --url, --port/--tls-port-options1-2] PASSED [ 70%] test/boardwalkd/test_boardwalkd_cli.py::test_incomplete_serve_command[Missing required parameters to serve boardwalkd: --port/--tls-port-options2-2] PASSED [ 73%] test/integration/test_workspaces.py::test_development_workspaces[ShouldSucceedTestWorkspace-False-] BECOME password: PASSED [ 76%] test/integration/test_workspaces.py::test_development_workspaces[ShouldSucceedPlaybookExecutionTestWorkspace-False-] PASSED [ 79%] test/integration/test_workspaces.py::test_development_workspaces[UITestVeryVeryLongWorkSpaceNameWorkspace-False-] PASSED [ 82%] test/integration/test_workspaces.py::test_development_workspaces[TaskJobWithOptionsShouldSucceedWorkspace-False-] PASSED [ 85%] test/integration/test_workspaces.py::test_development_workspaces[PlaybookJobWithOptionsShouldSucceedWorkspace-False-] PASSED [ 88%] test/integration/test_workspaces.py::test_development_workspaces[TaskJobWithPreconditionsShouldSucceedIfHostIsMacOSXWorkspace-False-] PASSED [ 91%] test/integration/test_workspaces.py::test_development_workspaces[TaskJobWithPreconditionsShouldBeSkippedIfHostIsMacOSXWorkspace-True-No hosts meet preconditions] PASSED [ 94%] test/integration/test_workspaces.py::test_development_workspaces[ShouldFailTestWorkspace-True-runner_on_failed: ansible.builtin.fail: ansible.builtin.fail: 127.0.0.1: Task failed successfully.: fatal: [127.0.0.1]: FAILED! => changed=false] PASSED [ 97%] test/integration/test_workspaces.py::test_development_workspaces[ShouldFailPlaybookExecutionTestWorkspace-True-runner_on_failed: Fail: ansible.builtin.fail: 127.0.0.1: Task failed successfully!: fatal: [127.0.0.1]: FAILED! => changed=false] PASSED [100%] ========================================================================================================= 34 passed in 317.84s (0:05:17) ========================================================================================================== poetry run ruff check All checks passed! poetry run ruff format --check 30 files already formatted export PYRIGHT_PYTHON_FORCE_VERSION=latest poetry run pyright WARNING: there is a new pyright version available (v1.1.350 -> v1.1.386). Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest` 0 errors, 0 warnings, 0 informations poetry run semgrep \ --config test/semgrep-rules.yml \ --config "p/r2c-security-audit" \ --config "p/r2c-bug-scan" \ --config "p/secrets" \ --config "p/dockerfile" ┌──── ○○○ ────┐ │ Semgrep CLI │ └─────────────┘ ⠹ Loading rules from registry...RPC response indicated an error: Error parsing RPC request: Atdgen_runtime.Oj_run.Error("Line 1:\nUnsupported variant \"CallValidate\"") Raised at Atdgen_runtime__Oj_run.error_with_line in file "atdgen-runtime/src/oj_run.ml", line 22, characters 2-18 Called from RPC.handle_single_request in file "semgrep/src/rpc/RPC.ml", line 131, characters 13-67 Failed to validate semgrep configuration ⠦ Loading rules from registry...RPC response indicated an error: Error parsing RPC request: Atdgen_runtime.Oj_run.Error("Line 1:\nUnsupported variant \"CallValidate\"") Raised at Atdgen_runtime__Oj_run.error_with_line in file "atdgen-runtime/src/oj_run.ml", line 22, characters 2-18 Called from RPC.handle_single_request in file "semgrep/src/rpc/RPC.ml", line 131, characters 13-67 Failed to validate semgrep configuration ⠸ Loading rules from registry...RPC response indicated an error: Error parsing RPC request: Atdgen_runtime.Oj_run.Error("Line 1:\nUnsupported variant \"CallValidate\"") Raised at Atdgen_runtime__Oj_run.error_with_line in file "atdgen-runtime/src/oj_run.ml", line 22, characters 2-18 Called from RPC.handle_single_request in file "semgrep/src/rpc/RPC.ml", line 131, characters 13-67 Failed to validate semgrep configuration ⠧ Loading rules from registry...RPC response indicated an error: Error parsing RPC request: Atdgen_runtime.Oj_run.Error("Line 1:\nUnsupported variant \"CallValidate\"") Raised at Atdgen_runtime__Oj_run.error_with_line in file "atdgen-runtime/src/oj_run.ml", line 22, characters 2-18 Called from RPC.handle_single_request in file "semgrep/src/rpc/RPC.ml", line 131, characters 13-67 Failed to validate semgrep configuration ⠦ Loading rules from registry...RPC response indicated an error: Error parsing RPC request: Atdgen_runtime.Oj_run.Error("Line 1:\nUnsupported variant \"CallValidate\"") Raised at Atdgen_runtime__Oj_run.error_with_line in file "atdgen-runtime/src/oj_run.ml", line 22, characters 2-18 Called from RPC.handle_single_request in file "semgrep/src/rpc/RPC.ml", line 131, characters 13-67 Failed to validate semgrep configuration Scanning 101 files (only git-tracked) with 539 Code rules: CODE RULES Language Rules Files Origin Rules ───────────────────────────── ─────────────────── 45 83 Community 321 python 169 47 Pro rules 216 yaml 1 3 Custom 2 dockerfile 5 1 SUPPLY CHAIN RULES 💎 Run `semgrep ci` to find dependency vulnerabilities and advanced cross-file findings. PROGRESS ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00 ┌──────────────┐ │ Scan Summary │ └──────────────┘ Some files were skipped or only partially analyzed. Scan was limited to files tracked by git. Scan skipped: 18 files matching .semgrepignore patterns For a full list of skipped files, run semgrep with the --verbose flag. Ran 220 rules on 83 files: 0 findings. ✨ If Semgrep missed a finding, please send us feedback to let us know! See https://semgrep.dev/docs/reporting-false-negatives/ (boardwalk-py3.12) asullivan@MBP-NT9RPG2XV7 boardwalk % ```

Checklist