[Improvement][plugin] K8sUtils.jobExist list all jobs in specified namespace, could cause high memory usage

ChaoquanTao commented 3 months ago

Search before asking

[X] I had searched in the issues and found no similar feature requirement.

Description

The org.apache.dolphinscheduler.plugin.task.api.utils.K8sUtils#jobExist lists all the job under specified namespace and then filter by name to judge whether the specified job exist. Because k8s doesn't delete completed job explicitly, listing all the job may cause a huge memory usage, especially when there's too many jobs.

Are you willing to submit a PR?

[X] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

SbloodyS commented 3 months ago

Hi, Would you like to fix it? @ChaoquanTao

ChaoquanTao commented 3 months ago

yeah sure, working on it

SbloodyS commented 3 months ago

yeah sure, working on it

I've assigned to you. Looking forward to your contribution.

apache / dolphinscheduler