blag commented 5 years ago

A ~gnarly Bash script~ awesome Python script to scan a pack repository for Mistral workflows, and convert as many workflows as possible. Attempts to rollback any changes if any part of the conversion process is not successful. Passes all unrecognized strings through to the orquestaconvert.sh script.

If you are in a pack repository, you run the orquestaconvert-pack.sh script - that's it.

Additionally, it includes a --list-workflows flag to list all of the workflows of a specific type.

I added tests for the --validate flag and removed them in another commit to try to make it easier to implement them in #13. There's a bit of a chicken and egg problem here regarding which PR (this one or #13) gets merged first.

actions/st2_pkg_e2e_test.meta.yaml --> actions/workflows/st2_pkg_e2e_test.yaml
actions/st2_pkg_test_and_promote_enterprise.meta.yaml --> actions/workflows/st2_pkg_test_and_promote_enterprise.yaml
actions/st2_pkg_upgrade_rpm.meta.yaml --> actions/workflows/st2_pkg_upgrade_rpm.yaml
actions/mistral.yaml --> actions/workflows/mistral.yaml
actions/st2_pkg_upgrade_deb.meta.yaml --> actions/workflows/st2_pkg_upgrade_deb.yaml
actions/st2_pkg_upgrade_e2e_test.meta.yaml --> actions/workflows/st2_pkg_upgrade_e2e_test.yaml
actions/st2_pkg_promote_all.meta.yaml --> actions/workflows/st2_pkg_promote_all.yaml
actions/st2_pkg_test_and_promote.meta.yaml --> actions/workflows/st2_pkg_test_and_promote.yaml
actions/bwc_pkg_promote_all.meta.yaml --> actions/workflows/bwc_pkg_promote_all.yaml

nmaludy commented 5 years ago

Would prefer this in python so we can have unit tests (and integration tests).

For integration tests, thinking of having a directory in test/fixtures that contains a few workflows. Should run this against that and make sure it does what we want.

Also, probably should add some documentation to the README and converting all of the actions for the pack.

blag commented 5 years ago

Just to document my own feature requests, after I port this to Python, I'd like to have it report a "roll up" of the unsupported features that workflows used. Something like this:

----- SUMMARY -----
Successfully converted:
  - st2_good_wf
  - st2_amazing_wf

Use the `with-items` attribute (and were not converted):
  - st2_wf_foo
  - st2_wf_bar

Use the `retry` attribute (and were not converted):
  - st2_wf_bar

Contain more than one workflow definition (and were not converted):
  - st2_wf_foo

codecov-io commented 5 years ago

Codecov Report

Merging #7 into master will decrease coverage by 1.55%. The diff coverage is 90.69%.

@@            Coverage Diff             @@
##           master       #7      +/-   ##
==========================================
- Coverage   99.71%   98.16%   -1.56%     
==========================================
  Files          10       11       +1     
  Lines         353      436      +83     
==========================================
+ Hits          352      428      +76     
- Misses          1        8       +7

Impacted Files	Coverage Δ
orquestaconvert/client.py	`97.36% <66.66%> (ø)`	:arrow_up:
orquestaconvert/pack_client.py	`91.56% <91.56%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update a93c1ac...3b8b729. Read the comment docs.

blag commented 5 years ago

Here's an example of the "roll up":

ERROR: Unable to convert all Mistral workflows.
ISSUE: Task 'clone_repos' contains an attribute 'with-items' that is not supported in orquesta.
Affected files:
  - actions/workflows/mistral_branch_candidate.yaml
  - actions/workflows/mistral_tag_release.yaml

ISSUE: Task 'promote_all_distros' contains an attribute 'with-items' that is not supported in orquesta.
Affected files:
  - actions/workflows/st2_release_packages.yaml
  - actions/workflows/bwc_release_packages.yaml

ISSUE: Task 'notify_success' contains an attribute 'with-items' that is not supported in orquesta.
Affected files:
  - actions/workflows/mistral_release_candidate.yaml

ISSUE: Task 'delete_volumes' contains an attribute 'with-items' that is not supported in orquesta.
Affected files:
  - actions/workflows/cleanup_volumes.yaml
  - actions/workflows/destroy_vm.yaml

ISSUE: Task 'package_all' contains an attribute 'with-items' that is not supported in orquesta.
Affected files:
  - actions/workflows/st2_stage_release_packages.yaml
  - actions/workflows/bwc_stage_release_packages.yaml

ISSUE: Task 'get_version_centos' contains an attribute 'concurrency' that is not supported in orquesta.
Affected files:
  - actions/workflows/st2_get_installed_version.yaml

ISSUE: Task 'wait_for_password_data' contains an attribute 'retry' that is not supported in orquesta.
Affected files:
  - actions/workflows/create_vm_windows.yaml

I'm not 100% sure I like that, but it's a good first step.

StackStorm / orquestaconvert

Add a script to convert an entire pack at once #7

Codecov Report