v3.8.0 release tracker

[nzlosh]

Pre-Release PR Reviews

@nzlosh has reviewed all issues in the v3.8 kanban board.

The following packages have been selected as release candidates for v3.8

• st2chatops package build st2chatops-3.8dev-4.
• st2web package build st2web-3.8dev-5.
• st2 package build st2-3.8-46.

Pre-release testing

• ubuntu 18.04 (bionic)
  • [x] bash installer succeeded
  • [x] st2-self-check succeeded
  • [x] manual st2web testing
• ubuntu 20.04 (focal)
  • [x] bash installer succeeded
  • [x] st2-self-check succeeded
  • [x] manual st2web testing
• centos7
  • [x] bash installer succeeded
  • [x] st2-self-check succeeded
  • [x] manual st2web testing
• centos8
  • [x] bash installer succeeded
  • [x] st2-self-check succeeded
  • [x] manual st2web testing

steps 4 and 5: create and e2e test the staging-stable packages

• ubuntu 18.04 (bionic)
  • [x] bash installer succeeded
  • [x] st2-self-check succeeded
  • [ ] manual st2web testing
• ubuntu 20.04 (focal)
  • [ ] bash installer succeeded
  • [ ] st2-self-check succeeded
  • [ ] manual st2web testing
• centos7
  • [x] bash installer succeeded
  • [x] st2-self-check succeeded
  • [x] manual st2web testing
• centos8
  • [x] bash installer succeeded
  • [x] st2-self-check succeeded
  • [x] manual st2web testing

step 6 staging-stable testing

• ubuntu 18.04 (bionic)
  • [ ] bash installer succeeded
  • [x] st2-self-check succeeded
  • [ ] manual st2web testing
• ubuntu 20.04 (focal)
  • [ ] bash installer succeeded
  • [x] st2-self-check succeeded
  • [ ] manual st2web testing
• centos7
  • [ ] bash installer succeeded
  • [x] st2-self-check succeeded
  • [ ] manual st2web testing
• centos8
  • [ ] bash installer succeeded
  • [ ] st2-self-check succeeded
  • [ ] manual st2web testing
• [ ] StackStorm upgrade from 3.7 to 3.8, following the Upgrade Instructions: https://docs.stackstorm.com/latest/install/upgrades.html#v3-8
  • [ ] Without optional data migration script
  • [ ] With optional data migration script - tested on smallish data set...
  • [ ] CentOS 7
  • [ ] CentOS 8
  • [ ] Ubuntu 18.04
  • [ ] Ubuntu 20.04

Major changes as detailed in next section...

If you have successful test results, please post a summary of what all you tested (OSes, what features you tested).

If you run into any bugs, please open them in the respective repositories and link to this issue from there. I will add them to the list at the bottom of this description.

If you have any issues running StackStorm or running the tests, please post down below.

step 7 stable testing

• ubuntu 18.04 (bionic)
  • [x] bash installer succeeded
  • [x] st2-self-check succeeded
  • [ ] manual st2web testing
• ubuntu 20.04 (focal)
  • [x] bash installer succeeded
  • [x] st2-self-check succeeded
  • [ ] manual st2web testing
• centos7
  • [x] bash installer succeeded
  • [x] st2-self-check succeeded
  • [ ] manual st2web testing
• centos8
  • [x] bash installer succeeded
  • [ ] st2-self-check succeeded
  • [ ] manual st2web testing

step 8 tag releases

• [x] st2client on pypi
• [x] st2 tag
• [x] packer-st2
• [x] api-docs (new /clone endpoint for actions is present, so this was updated)
• [x] docker

step 9 update st2cd and st2ci packs

step 10 create gtihub release of st2 repository

https://github.com/StackStorm/st2/releases/tag/v3.8.0

step 11 make sure new dev builds succeed

• [x] https://app.circleci.com/pipelines/github/StackStorm/st2
• [x] https://app.circleci.com/pipelines/github/StackStorm/st2docs
• [ ] https://app.circleci.com/pipelines/github/StackStorm/st2web
• [x] https://app.circleci.com/pipelines/github/StackStorm/st2chatops

step 12 Check PackageCloud for new packages in unstable channel

• [x] u18
• [x] u20
• [x] el8
• [x] el7

step 13 Prep development version on master branch of st2-dockerfiles

step 14 Ensure that the AWS instances EC2 instances are all stopped:

• [ ] ubuntu-build-itest001
• [ ] ubuntu-build002
• [ ] ubuntu-build003

step 15 Delete the datastore keys

step 16 Update the version numbers in steps 3, 12, and 13 for the next release manager

blog article

• [x] draft
• [x] review
• [ ] publish

Close v3.8.0

• [ ] close this issue
• [ ] close https://github.com/orgs/StackStorm/projects/29

Major changes

• [ ] St2 communicates with redis over TLS connections.
• [ ] St2 processes patternProperties and additionalItems defined in pack schema.
• [ ] Improved nested array support in schemas.
• [ ] HTTP_PROXY/HTTPS_PROXY environment variables are checked and used by st2client.
• [ ] Workflow engine graceful shutdown.
• [ ] BREAKING CHANGE workflow output schema validation (disable by default). Invalid output schemas are silently ignored.
• [ ] Date/time module switched from udatetime to ciso8601.
• [ ] Deprecated st2exporter removed from st2.
• [ ] hubot-slack adpater updated to 4.10.0.
• [ ] Synchronous responses button added to webui.

st2

Fixed

• Fix redis SSL problems with sentinel #5660

• Fix a bug in the pack config loader so that objects covered by an patternProperties schema or arrays using additionalItems schema(s) can use encrypted datastore keys and have their default values applied correctly. #5321

  Contributed by @cognifloyd

• Fixed st2client/st2client/base.py file to check for http_proxy and https_proxy environment variables for both lower and upper cases.

  Contributed by @S-T-A-R-L-O-R-D

• Fixed a bug where calling 'get_by_name' on client for getting key details was not returning any results despite key being stored. #5677

  Contributed by @bharath-orchestral

• Fixed st2client/st2client/base.py file to use https_proxy(not http_proxy) to check HTTPS_PROXY environment variables.

  Contributed by @wfgydbu

• Fixed schema utils to more reliably handle schemas that define nested arrays (object-array-object-array-string) as discovered in some of the ansible installer RBAC tests (see #5684). This includes a test that reproduced the error so we don't hit this again. #5685

• Fixed eventlet monkey patching so more of the unit tests work under pytest. #5689

• Fix and reenable prance-based openapi spec validation, but make our custom x-api-model validation optional as the spec is out-of-date. #5709 Contributed by @cognifloyd

• Fixed generation of st2.conf.sample to show correct syntax for [sensorcontainer].partition_provider (space separated key:value pairs). #5710 Contributed by @cognifloyd

Added

• Added graceful shutdown for workflow engine. #5463 Contributed by @khushboobhatia01

• Add ST2_USE_DEBUGGER env var as alternative to the --use-debugger cli flag. #5675 Contributed by @cognifloyd

• Added purging of old tokens. #5679 Contributed by Amanda McGuinness (@amanda11 intive)

• Begin introducing pants <https://www.pantsbuild.org/docs>_ to improve DX (Developer Experience) working on StackStorm, improve our security posture, and improve CI reliability thanks in part to pants' use of PEX lockfiles. This is not a user-facing addition. #5713 #5724 #5726 #5725 #5732 #5733 #5737 Contributed by @cognifloyd

Changed

• BREAKING CHANGE for anyone that uses output_schema, which is disabled by default. If you have [system].validate_output_schema = True in st2.conf AND you have added output_schema to any of your packs, then you must update your action metadata.

  output_schema must be a full jsonschema now. If a schema is not well-formed, we ignore it. Now, output can be types other than object such as list, bool, int, etc. This also means that all of an action's output can be masked as a secret.

  To get the same behavior, you'll need to update your output schema. For example, this schema:

    output_schema:
      property1:
        type: bool
      property2:
        type: str

should be updated like this:

    output_schema:
      type: object
      properties:
        property1:
          type: bool
        property2:
          type: str
      additionalProperties: false

5319

Contributed by @cognifloyd

• Changed the X-XSS-Protection HTTP header from 1; mode=block to 0 in the conf/nginx/st2.conf to align with the OWASP security standards. #5298

  Contributed by @LiamRiddell

• Use PEP 440 direct reference requirements instead of legacy PIP VCS requirements. Now, our *.requirements.txt files use package-name`@ git+https://url@version ; markers instead of git+https://url@version#egg=package-name ; markers. #5673 Contributed by@cognifloyd`

• Move from udatetime to ciso8601 for date functionality ahead of supporting python3.9 #5692 Contributed by Amanda McGuinness (@amanda11 intive)

• Refactor tests to use python imports to identify test fixtures. #5699 #5702 #5703 #5704 #5705 #5706 Contributed by @cognifloyd

• Refactor st2-generate-schemas so that logic is in an importable module. #5708 Contributed by @cognifloyd

Removed

• Removed st2exporter service. It is unmaintained and does not get installed. It was originally meant to help with analytics by exporting executions as json files that could be imported into something like elasticsearch. Our code is now instrumented to make a wider variety of stats available to metrics drivers. #5676 Contributed by @cognifloyd

st2chatops

Added

• A changelog file.

  Contributed by @nzlosh

Fixed

• Fix the CI build caused by different OS version.

• Fix CircleCI build for forked PRs.

  Contributed by @armab

Changed

• Updated hubot-slack to "^4.10.0"

  Contributed by @lopf

Removed

• Remove skills-validator from npm-shrinkwrap.json

  Contributed by @armab

st2web

Added

• Added feature for disabling button for synchronous responses

  • Button gets disabled onClick on Connect and Submit in st2-login and st2-history module respectively.

  Contributed by @ParthS007

Fixed

• Fixed CircleCI tests

  Contributed by @amanda11

• Handle cases where content-type is not only application/json

  Contributed by @luislobo

Conclusion

Please report findings here and bugs/regressions in respective repositories.

Depending on severity and importance bugs might be fixed before the release or postponed to the next release if they're very minor and not a release blocker.

Issues Found During Release

• to be determined

TODOs

• [ ] Blog post for release
• [ ] Blog post for exchange/community update

[arm4b]

For someone willing to help with the pre-release testing, use this to install the latest 3.8dev branch on a new VM:

bash <(curl -sSL https://stackstorm.com/packages/install.sh) --user=st2admin --password=Ch@ngeMe --staging --unstable

Try out features and changes that landed in this version (the first message ^^) for any regressions or issues. Please let us know how it goes.

[amanda11]

@nzlosh Performed some upgrade testing on Rocky8 from 3.7->3.8. New 3.7 installation with a few pack installs, aliases, and slack st2chatops connectivity. Upgrade went through successfully. After upgrade did some st2web testing on 3.8 on EL8 and all good.

[amanda11]

@nzlosh Performed some upgrade testing on Centos7 from 3.7->3.8. Prior to upgrade had a workflow that had the old style output schema with a secret. After upgrade (without changing the schema), checked that running workflow that the secret was still masked - and it was, but other validation didn't work (as expected as still using old style schema). Updated pack to use new style schema - and validation succeeded. After upgrade did some testing with st2web on 3.7, mainly auto-save on the workflow composer. No issues found.

[cognifloyd]

We can drop "Removed skills-validator for hubot." from the list of major features.

We have to remove that every time we regenerate npm-shrinkwrap.json. I don't know why npm keeps adding it, but it breaks CI.

https://github.com/StackStorm/st2chatops/commits/master/npm-shrinkwrap.json

[nzlosh]

Sure, it was questioned earlier today too.

[arm4b]

FYI the packages were promoted by the Release Team (@nzlosh @rush-skills) from staging-unstable -> staging-stable. If someone willing to take a final check before the packages will go to production, here is the one-line install:

bash <(curl -sSL https://stackstorm.com/packages/install.sh) --staging --stable --user=st2admin --password=Ch@ngeMe