apache / incubator-devlake

Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering excellence, developer experience, and community growth.
https://devlake.apache.org/
Apache License 2.0
2.51k stars 493 forks source link

[Bug][PagerDuty] not enough info for Pagerduty execution (400) #7561

Closed yoramshai closed 1 month ago

yoramshai commented 1 month ago

Search before asking

What happened

After upgrade to v1.0.0-beta9 PagerDuty tasks stopped to work? error preparing task data for pagerduty (400) Wraps: (2) not enough info for Pagerduty execution (400) Error types: (1) *hintdetail.withDetail (2) *errors.errorString

What do you expect to happen

Pipeline will be completed

How to reproduce

In my case if was upgrade from v1.0.0-beta6 --> v1.0.0-beta9

Anything else

No response

Version

v1.0.0-beta9

Are you willing to submit PR?

Code of Conduct

dosubot[bot] commented 1 month ago

Hey @yoramshai! 👋 I'm here to help you with any bugs, questions, or contributions you have regarding this project. Let's squash those bugs together!

The error message you're encountering after upgrading to v1.0.0-beta9 is likely due to changes in the Incident struct, which introduced new fields that might not be handled correctly in your existing codebase. Specifically, the following fields were added to the Incident struct in v1.0.0-beta9:

Ensure that your task options are correctly decoded and validated. The PagerDutyOptions struct and related functions in task_data.go should properly handle the new fields and any required parameters. Here is an example of how the task options are managed:

type PagerDutyOptions struct {
    ConnectionId  uint64                       `json:"connectionId" mapstructure:"connectionId,omitempty"`
    ServiceId     string                       `json:"serviceId,omitempty" mapstructure:"serviceId,omitempty"`
    ServiceName   string                       `json:"serviceName,omitempty" mapstructure:"serviceName,omitempty"`
    ScopeConfigId uint64                       `json:"scopeConfigId,omitempty" mapstructure:"scopeConfigId,omitempty"`
    ScopeConfig   *models.PagerdutyScopeConfig `json:"service_name,omitempty" mapstructure:"serviceName,omitempty"`
}

type PagerDutyTaskData struct {
    Options *PagerDutyOptions
    Client  api.RateLimitedApiClient
}

func DecodeAndValidateTaskOptions(options map[string]interface{}) (*PagerDutyOptions, errors.Error) {
    op, err := DecodeTaskOptions(options)
    if err != nil {
        return nil, err
    }
    err = ValidateTaskOptions(op)
    if err != nil {
        return nil, err
    }
    return op, nil
}

func DecodeTaskOptions(options map[string]interface{}) (*PagerDutyOptions, errors.Error) {
    var op PagerDutyOptions
    err := api.Decode(options, &op, nil)
    if err != nil {
        return nil, err
    }
    return &op, nil
}

func EncodeTaskOptions(op *PagerDutyOptions) (map[string]interface{}, errors.Error) {
    var result map[string]interface{}
    err := api.Decode(op, &result, nil)
    if err != nil {
        return nil, err
    }
    return result, nil
}

func ValidateTaskOptions(op *PagerDutyOptions) errors.Error {
    if op.ServiceName == "" {
        return errors.BadInput.New("not enough info for Pagerduty execution")
    }
    if op.ServiceId == "" {
        return errors.BadInput.New("not enough info for Pagerduty execution")
    }
    if op.ConnectionId == 0 {
        return errors.BadInput.New("connectionId is invalid")
    }
    return nil
}

Ensure that your code handles these new fields appropriately to avoid errors [1][2].

To continue talking to Dosu, mention @dosu.

klesh commented 1 month ago

Hi, can you share the plan detail of the failed pipeline? image

yoramshai commented 1 month ago

@klesh absolutely
{ "id": 632, "name": "foo-Blueprint", "plan": [ [ { "plugin": "org", "subtasks": [ "setProjectMapping" ], "options": { "projectMappings": [ { "projectName": "foo", "scopes": [ { "rowId": "github:GithubRepo:1:569576293", "table": "repos" }, { "rowId": "github:GithubRepo:1:569576293", "table": "cicd_scopes" }, { "rowId": "github:GithubRepo:1:569576293", "table": "boards" }, { "rowId": "pagerduty:Service:1:P5GD7S8", "table": "cicd_scopes" }, { "rowId": "webhook:27", "table": "cicd_scopes" }, { "rowId": "webhook:27", "table": "boards" } ] } ] } } ], [ { "plugin": "github_graphql", "subtasks": [ "Collect Milestones", "Extract Milestones", "Collect Issues", "Extract Issues", "Collect Pull Requests", "Extract Pull Requests", "Collect Workflow Runs", "Extract Workflow Runs", "Collect Job Runs", "Extract Jobs", "Collect Comments", "Extract Comments", "Collect Events", "Extract Events", "Collect PR Review Comments", "Extract PR Review Comments", "Collect Users", "Extract Users", "Convert Workflow Runs", "Convert Jobs", "Enrich PR Issues", "Convert Repos", "Convert Issues", "Convert Issue Labels", "Convert PR Commits", "Convert Pull Requests", "Convert PR Reviews", "Convert PR Labels", "Convert PR Issues", "Convert Issue Assignees", "Convert Issue Comments", "Convert PR Comments", "Convert Milestones", "Convert Users", "Collect Deployments", "Extract Deployments", "Convert Deployments", "Collect Releases", "Extract Releases", "Convert Releases" ], "options": { "connectionId": 1, "fullName": "bar/foo", "githubId": 569576293, "name": "bar/foo" } }, { "plugin": "gitextractor", "subtasks": null, "options": { "fullName": "bar/foo", "name": "bar/foo", "proxy": "", "repoId": "github:GithubRepo:1:569576293", "url": "https://git:****************************************@github.com/bar/foo.git" } }, { "plugin": "pagerduty", "subtasks": [ "collectIncidents", "extractIncidents", "convertIncidents", "convertServices" ], "options": { "connectionId": 1, "serviceId": "P5G****" } } ], [ { "plugin": "dora", "subtasks": [ "generateDeployments", "generateDeploymentCommits", "enrichPrevSuccessDeploymentCommits" ], "options": { "projectName": "foo" } } ], [ { "plugin": "refdiff", "subtasks": [ "calculateDeploymentCommitsDiff" ], "options": { "projectName": "foo" } } ], [ { "plugin": "dora", "subtasks": [ "calculateChangeLeadTime", "ConnectIncidentToDeployment" ], "options": { "projectName": "foo" } } ] ], "skipOnFail": true }