WeBankFinTech / DataSphereStudio

DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.
https://github.com/WeBankFinTech/DataSphereStudio-Doc
Apache License 2.0
3.09k stars 1k forks source link

[Feature] Clean up downloaded lib packages when the appconn engine exits #1085

Closed wxyn closed 1 year ago

wxyn commented 1 year ago

Search before asking

Problem Description

After the appconn engine exits, the downloaded appconn packages will not be automatically cleaned up in the engine directory, resulting in an increasing disk space usage. I hope that after optimization, the lib package downloaded by appconn can be automatically deleted when the engine exits.

Description

Due to the fact that the dss-appconns directory is not softlinked to public resources, each engine will download a new directory, resulting in disk usage. Therefore, when the appconn engine exits, it is necessary to clean the dss appconns directory.

Use case

No response

solutions

  1. Rewrite the hook method (ComputationExecutor. tryShutdown()) when the linkis engine exits in the AppConnEngineConnExecutor class, and add appconn's lib directory cleaning logic to this method.
  2. Reuse the AppConnUtils. getAppConnHomePath() method of existing tool classes to obtain the root directory of the appconn engine

Anything else

No response

Are you willing to submit a PR?

zqburde commented 1 year ago

Add it in DSS1.1.2