Standardize scripts/tasks names and execution order on KIE Tools packages

[thiagoelg]

This task aims to define the scripts' names and functionalities. For example:

• Every buildable package should have a build:dev and build:prod script, the built files should be inside <package_name>/dist;
• Every unit testable package should have a test script, test results should be stored in <package_name>/dist-tests; and so on...

It's also expected to have an execution order defined. For example:

1. Install dependencies;
2. Build;
3. Test;
4. and so on...

This is important so that our monorepo is more readable and manageable and makes it easier for future improvements, such as build and test caches.

[tiagobento]

Phase 1: Prepare (Implementing conventions)

Scripts will be conventionalized, but build:dev and build:prod will still invoke other scripts.

• Top-level

  • bootstrap [pnpm-filter?]

    Invokes pnpm install, creating and populating node_modules dirs on each package. Invokes install script on each package individually, topologically ordered. Takes an optional positional argument that is a pnpm filter.

  • format:all

    Formats the whole code base.

  • format:changed

    Formats uncommitted files.

  • format:check

    Checks if all files match formatting rules. Used by PR checks and the CI.

  • update-version-to [new-version] [pnpm-filter?]

    Updates package.json --> version to [new-version]. Must be SemVer compatible. Takes an optional positional argument that is a pnpm filter. ℹ️ Runs bootstrap at the end.

  • update-stream-name-to [new-name]

    Conventionalized to follow the development stream branch name. E.g., main or 10.0.x, 10.1.x etc. Used by Container images as tag.

  • update-kogito-version-to [new-version]

    Updates the Kogito version. This is a Maven artifact version. E.g., 999-SNAPSHOT or 999-20240509-SNAPSHOT or 10.0.0.

• Packages

  • install

    Sets versions, download dependencies, and does whatever one-time things it has to Called automatically by the top-level bootstrap script.

  • clean

    Removes output directories. Usually dist or **/target

  • lint

    Performs source code analysis 🚩 Toggled by $(build env linters.run)

  • build:dev

    Builds for development. No optimizations or multiple compilation targets. Must provide everything for dependent packages, though. Will most likely invoke clean;

  • build:prod

    Builds for production. Will invoke clean, lint, test and test-e2e.

  • test

    Runs unit tests. Should be fast and lightweight. 🚩 Toggled by $(build-env tests.run) ❎ Affected by $(build-env tests.ignoreFailures)

  • test-e2e

    Runs end-to-end tests. Can spin up other apps and heavier components. 🚩 Toggled by $(build-env endToEndTests.run) ❎ Affected by $(build-env endToEndTests.ignoreFailures)

  • start

    Runs the package for development.

Phase 2: Turbo (independent scripts)

By using turbo, we'll be able to make all scripts independent. Their relationship will be defined at the top-level turbo.json. build:dev and build:prod won't invoke other scripts anymore.

• turbo [top-level-script]

  Will work the same way as before, just turbo will invoke them instead of pnpm. [pnpm-filter?] becomes [turbo-filter?]

• turbo clean build:dev [turbo-filter?]

  Will clean and build for development.

• turbo clean build:prod [turbo-filter?]

  Will clean, and build for production. Note how lint, test and test-e2e are not ran. Note how test and test-e2e are not executed.

• turbo clean start [turbo-filter?]

  Will run start for all packages filtered by [turbo-filter]. No more multiple tabs!

• turbo clean lint build:prod test test-e2e [turbo-filter?]

  Will do everything. No more flags for toggling linters and tests.

Phase 3: Wrap-up (Cleanup)

All non-conventionalized scripts will be renamed to ~:[previous-name]. This will group them together at the end and make all package.json files similar. Reducing the cognitive load to understand package.json --> scripts that have a lot of tasks inside.

E.g.:

{
  ...
  "scripts": {
    "build:dev": "pnpm ~:copy:css && tsc -p tsconfig.json",
    "build:prod": "pnpm ~:copy:css && tsc -p tsconfig.json",
    "clean": "rimraf 'dist*'",
    "lint": "kie-tools--eslint ./src",
    "start": "cross-env STORYBOOK_PORT=$(build-env dmnEditor.storybook.port) pnpm storybook-base --storybookArgs=\"dev --no-open\"",
    "test": "run-script-if --true --ignore-errors \"$(build-env tests.ignoreFailures)\" --then \"jest --silent --verbose --passWithNoTests\"",
    "test-e2e": "run-script-if --true --ignore-errors \"$(build-env endToEndTests.ignoreFailures)\" --then \"pnpm exec playwright test\"",
    "~:build:storybook": "storybook build -o dist-storybook",
    "~:copy:css": "copyfiles -u 1 \"src/**/*.{sass,scss,css}\" dist/",
    "~:test-e2e:open": "pnpm exec playwright show-report dist-tests-e2e/reports"
  },
  ...
}

[tiagobento]

After some research, I think we should postpone Phase 2: Turbo for a while. It's a big shift and I think we need more time for everyone to be comfortable with the current setup before we introduce a change this disruptive to the usage of kie-tools.