[WIP] KCL configuration files watch system design

[changsun20]

Overview

The KCL dependency watch system is a component designed to monitor and manage dependencies within KCL projects. It is inspired by the efficiency and user-friendliness of tools like Cargo for Rust, aiming to provide a seamless development experience by handling real-time dependency updates and integration with the KCL language server.

Note: This issue serves as a work-in-progress outline for the design and implementation of the watch system.

Previous issues and discussions

• #998
• #1118
• #1212

Design Goals

1. Real-time Feedback: Provide immediate updates on dependency changes.
2. Automation: Automatically handle dependency installation, updates, and removal based on configuration changes.
3. Integration: Seamlessly integrate with the KCL language server for a cohesive development environment.
4. User Experience: Mimic the ease of use and reliability of existing tools like Cargo.

Core Components

1. Watcher

• Utilize a file system notification library (notify, to be specific) to monitor changes in the project directory.
  • cargo-watch is built upon the notify library (see here)
• Operate asynchronously for responsiveness.

2. File Type and Change Detector

• Analyze file extensions to determine file types, and map them to corresponding handlers.

3. Handler Registry

• Manage operations for different tasks.
  • Resolve the information contained in the edited configuration files, compare it with the dependencies in current environment, and check if further operations are needed.
  • Trigger installation event, and run kcl mod add command.
  • Trigger update event, and run kcl mod update command.
• The handlers can be extended to more general usage for other .json and .yaml files in the future.

4. Configuration Manager (optional for users, in order to lower the cost of releases)

• Manage settings for the watch system, allowing customization through configuration files.
• Possible configurations:
  • Decide the types of watching files (.mod, .k, .json, .yaml, etc.), and the logic for different configuration files.
  • Decide the paths of watching configuration files.

Example Workflow: Dependency Update on Configuration Change

1. KCL Language Server main loop starts and initializes the watcher. The watcher automatically detect the kcl.mod file in the project, and start monitoring for changes.
2. Developer modifies kcl.mod to add/update a dependency.
3. Watcher detects file change and triggers an event.
4. File Type Detector identifies type of the changed configuration file, and start the handlers.
5. Handler Registry calls for an handler to resolve the kcl.mod file and compare it with the packages in working environment, which turned out that the dependency has not been installed yet. It then triggers another add/update event, and runs kcl mod add/update xxx command.
6. CLI runs the right command, and the task is finished.

Challenges

1. Efficient Dependency Management: Ensuring that dependencies are only updated when necessary, without redundant operations.
2. Configuration File Parsing: Accurately interpreting kcl.mod to manage dependencies effectively.
3. Error Handling: Providing clear, actionable feedback for configuration errors or dependency conflicts.

Proposed Solutions

• Implement incremental compilation checks to avoid unnecessary compiling.
• Develop/Utilize a robust parser for kcl.mod to ensure accurate dependency management.
• Integrate with the IDE to provide real-time error detection and resolution suggestions.

[Peefy]

Good Job! @changsun20 Thanks for the proposal. ❤️

cc @zong-zhe @He1pa Could you help review it?

[He1pa]

Manage operations for different tasks. Resolve the information contained in the edited configuration files, compare it with the dependencies in current environment, and check if further operations are needed.

• Trigger installation event, and run kcl mod add command.
• Trigger update event, and run kcl mod update command.

what means of installation and update event? It is user action of add/update dependencies in kcl.mod, or event in language server state? If so, kcl mod update would be more appropriate

[zong-zhe]

Hi @changsun20 😄

You can also add some design details, including:

1. API design: Watch System should be called to KCL LSP as a tripartite library, giving an API or abstract interface.

2. The contents of the Watch System should be divided into some subtasks, and the corresponding time point should be clearly defined. If it cannot be divided, the time point of the overall task can be given.

3. The Watch System and the integration of kpm and IDE are all the work, and the integration should be considered together with the watch system when planning the time schedule.

[changsun20]

what means of installation and update event? It is user action of add/update dependencies in kcl.mod, or event in language server state? If so, kcl mod update would be more appropriate

Well, I think my initial thought was user action of add/update. So yes you are write, the language here is not precise enough.

1. API design: Watch System should be called to KCL LSP as a tripartite library, giving an API or abstract interface.

2. The contents of the Watch System should be divided into some subtasks, and the corresponding time point should be clearly defined. If it cannot be divided, the time point of the overall task can be given.

3. The Watch System and the integration of kpm and IDE are all the work, and the integration should be considered together with the watch system when planning the time schedule.

Sure, I'll handle these updates ASAP.