Remove the golden.yaml files in the konfig repo

[amyXia1994]

Background

Those golden files are used mainly for the smoking test, listing affected stacks and displaying the expected YAML diffs in the PR CI progress. But the downsides are appearing:

• The golden files costumes lots of storage and slows down code fetching from GitHub: line count: 237w(golden.yaml) vs 70w(.k); file count: 8449(golden.yaml) vs 13381(.k)
• users must locally re-compile all the affected stacks to the corresponding stdout.golden.yaml files. That's time-consuming. And it's also not evident to users who are first to the Kusion stack.
• during the ci test, the golden files must be checked with the compiled output in the CI. That's time-consuming. And if users always compile locally, the CI process is barely checking if the output compiled in during CI is exactly the same as that compiled by users' local machine. And that's not very meaningful for the configuration correctness.
• the committed golden files are easily conflict when there are multiple modifications on the stack and the base schemas in parallel.

For all the above reasons, The golden files need to be removed from the repo, on the premise that all the usages of them are reserved:

• changes in files cannot break the compilability. If so, that change should be blocked in the CI process.
• Users can review the low-level data changes that the code change brings: The CI process should list all the affected stacks and display the data diffs that the PR brings.

Furthermore, the stability of the code in konfig still needs to be enhanced by unit and end-to-end tests. That will be discussed in issue: todo

Solution

Tools

Update: kusion compile

Current

usage is:kusion compile -w <stackPath>. When the -o option is not specified, the command will by default output to the ci-test/stdout.golden.yaml file.

Update to

• by default output to the stdout stream.
• add a --front option to output the frontend configuration.

Optimize: kusion deps

The kusion deps tools should be optimized to identify the configuration changes more precisely, so to reduce the re-compile time, then improve the local and the CI efficiency.

Current

The dependency analysis is based on the import relationship between files.

Update to

The kusion deps tool should support attribute-level dependency analysis.

Update: CI

The CI pipeline needs to support listing affected stacks and the diff details:

• list affected stacks
• show diffs of affected stacks

Update: Konfig script

• remove the check-App script
• support viewing Yaml format diff of specific app configuration
• support listing changed stacks after modifying the basic logic.

Add: Kusion IDE

Kusion IDE needs to provide convenient user interaction based on the above tools:

live preview of app configuraiton's Yaml representation

the feature is detailed in: https://github.com/KusionStack/vscode-kcl/issues/14

Provide a preview Yaml output panel when the user's active editor is at a base.k/main.k file under a stack:

• the Yaml output preview could be live according to the KCL code modification.
• the live preview supports either frontend preview or backend preview to satisfy the requirements of both app owners and lib maintainers
• when the user is adopting a source control, the live preview could provide the highlighted diff of the Yaml data according to changes on the source code.
• asynchronously check the affected stacks' compilability. And before the user commits to the source control, use the pre-commit hook to check if the app configurations within all the stacks are valid.

filter and focus on the changed stacks to provide a local review

Provide filters when displaying the konfig apps:

• based on the dependency relationship to filter the stacks. the upstream could be user-defined or from the git diff change.
• based on the app name, or the app type, or the organization architecture, ..., to filter the stacks

change review panel

If the repo is adopting a source control, the Kusion IDE needs to add a panel to display the changed stack list and a left-to-right Yaml diff of stacks, so users could review these changes before committing.

The related components need to be updated

There are some components that are aware of or rely on the stdout.golden.yaml files. They need to be updated before the golden files are finally removed from the konfig repo:

• The New App Releasement component
• The Kusion server
• The ci component and scripts
• The locally used scripts in konfig repo
• The documents that are spread over many teams and scenarios
• The examples to use kusion
• ats platforms

During the migration, deprecated warnings could be widely used to inform users to migrate to the new process. And users should be announced before the changes are public released.

Phased Goals

stage 1:

• Kusion IDE: support preview YAML data: frontend & backend
• kusion compile: remove default output to golden
• ci pipeline: show YAML diff in stacks between the target branch and the source branch

stage 2:

• optimize the kusion deps tool to support attribute-level dependency analysis.
• Kusion IDE: support list stack diffs triggered by certain changes.

[Peefy]

U can move this issue into the konfig repo.

[jakezhu9]

I completely agree with you! Your suggestion about improving tests and enhancing user experience is fantastic.

By the way, are there any updates or progress regarding this issue? I believe it's important and we should try to speed up the process.