[Bug][gitextractor] Unable to disable CollectDeployment

[lukayeh]

Search before asking

• [X] I had searched in the issues and found no similar issues.

What happened

i’d like to disable the gitlab-collectDeployment but having difficulties doing so via environments variables:

                "plugin": "gitlab",
                "subtasks": [
                    "CollectDeployment",

this is what I’m using: ENABLE_SUBTASKS_BY_DEFAULT="gitlab:CollectDeployment:false"

What do you expect to happen

CollectDeployments phase is removed from transform.

How to reproduce

set: ENABLE_SUBTASKS_BY_DEFAULT="gitlab:CollectDeployment:false

Anything else

we had a look at the source (even v1.0.0-beta6) the Gilab plugin blueprint creation bypasses this function: https://github.com/apache/incubator-devlake/blob/v1.0.0-beta6/backend/helpers/pluginhelper/api/pipeline_plan.go#L60 The above function utilises the ENABLE_SUBTASKS_BY_DEFAULT to enable or disable subtasks for the plugin. No other part of the code utilises that environment variable.

The Gitlab plugin bypasses MakePipelinePlanTask in the helper by calling MakePipelinePlanSubtasks helper instead which is what is ultimately called by MakePipelinePlanTask: https://github.com/apache/incubator-devlake/blob/v1.0.0-beta6/backend/plugins/gitlab/api/blueprint_v200.go#L127

Version

v0.21.0@ac4f7fe

Are you willing to submit PR?

• [ ] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[dosubot[bot]]

Hello @lukayeh! I'm here to help you with any bugs, questions, and contributions for the DevLake project. Let me know how I can assist you!

The issue you're encountering is due to the GitLab plugin directly calling MakePipelinePlanSubtasks instead of MakePipelinePlanTask, which processes the ENABLE_SUBTASKS_BY_DEFAULT environment variable.

To address this, you can modify the GitLab plugin's blueprint creation process to use MakePipelinePlanTask instead of MakePipelinePlanSubtasks. This change will ensure that the ENABLE_SUBTASKS_BY_DEFAULT environment variable is respected.

Here's a potential modification to the blueprint_v200.go file:

// Replace the direct call to MakePipelinePlanSubtasks with MakePipelinePlanTask
subtasks, err := helper.MakePipelinePlanTask("gitlab", subtaskMetas, scopeConfig.Entities, options)
if err != nil {
    return nil, err
}

This change will ensure that the CollectDeployment subtask can be disabled using the ENABLE_SUBTASKS_BY_DEFAULT environment variable.

After making this change, you can set the environment variable as follows:

export ENABLE_SUBTASKS_BY_DEFAULT="gitlab:CollectDeployment:false"

Then, restart the DevLake service to apply the changes:

• For Docker Compose, run docker-compose down and docker-compose up -d.
• For Helm, run helm upgrade devlake devlake/devlake --recreate-pods.

This should remove the CollectDeployment phase from the transformation process as expected ^[1][2].

To continue talking to Dosu, mention @dosu.