[Bug][Projects] Projects API POST endpoint does not create blueprint connections

jensrotne commented 1 week ago

Search before asking

[X] I had searched in the issues and found no similar issues.

What happened

When I try to create a new project with the POST /api/projects endpoint, then even if I include the blueprint field in the JSON body, then it doesn't correctly create the blueprint connections.

I was able to do a workaround where I create the project using the POST endpoint, fetch the blueprint ID from the response, and then use the PATCH /api/blueprints/{blueprint_id} to correctly set the connections.

What do you expect to happen

I expect that when putting a complete Project payload with a Blueprint field, that it will fully create the blueprint and it's dependencies as well.

How to reproduce

Create a new Project using the POST /api/projects endpoint with a a blueprint containing connections.

Anything else

The following issue exists, but is not classified as a bug: https://github.com/apache/incubator-devlake/issues/7610

Version

v1.0.0-rc1

Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

dosubot[bot] commented 1 week ago

@jensrotne Hello there! I'm here to help you out with any bugs, questions, or contributions you might have for the Apache DevLake project. Let me know how I can assist you!

The issue with the POST /api/projects endpoint not correctly creating blueprint connections when including the blueprint field in the JSON body is due to improper handling of the Connections field in the CreateBasicBlueprintV2 function. The Connections field should be a slice of *models.BlueprintConnection, but it is being set to a single config.Connection.

To fix this, ensure that the Connections field is correctly populated with the appropriate connections. Here is the corrected code snippet for the CreateBasicBlueprintV2 function:

func (d *DevlakeClient) CreateBasicBlueprintV2(name string, config *BlueprintV2Config) models.Blueprint {
    blueprint := models.Blueprint{
        Name:        name,
        ProjectName: config.ProjectName,
        Mode:        models.BLUEPRINT_MODE_NORMAL,
        Plan:        nil,
        Enable:      true,
        CronConfig:  "manual",
        IsManual:    true,
        SyncPolicy: models.SyncPolicy{
            SkipOnFail: config.SkipOnFail,
        },
        Labels: []string{"test-label"},
        Connections: []*models.BlueprintConnection{
            config.Connection, // Ensure this is correctly populated
        },
    }
    d.testCtx.Helper()
    blueprint = sendHttpRequest[models.Blueprint](d.testCtx, d.timeout, &testContext{
        client:       d,
        printPayload: true,
        inlineJson:   false,
    }, http.MethodPost, fmt.Sprintf("%s/blueprints", d.Endpoint), nil, &blueprint)
    return blueprint
}

Make sure that config.Connection is correctly set to the appropriate connection details before calling this function ^[1].

Additionally, ensure that the Post function in the blueprints package correctly handles the creation of blueprints. This function binds the JSON body to a models.Blueprint object and calls services.CreateBlueprint to handle the creation logic:

func Post(c *gin.Context) {
    blueprint := &models.Blueprint{}
    err := c.ShouldBind(blueprint)
    if err != nil {
        shared.ApiOutputError(c, errors.BadInput.Wrap(err, shared.BadRequestBody))
        return
    }
    err = services.CreateBlueprint(blueprint)
    if err != nil {
        shared.ApiOutputError(c, errors.Default.Wrap(err, "error creating blueprint"))
        return
    }
    shared.ApiOutputSuccess(c, blueprint, http.StatusCreated)
}

Ensure that the services.CreateBlueprint function correctly processes the Connections field and other blueprint details ^[2].

To continue talking to Dosu, mention @dosu.

klesh commented 1 week ago

Weird, it has been tested. Would you share your request in curl or any other executable format to reproduce the problem?

klesh commented 1 week ago

It should be working according to https://github.com/apache/incubator-devlake/pull/7449

kamiyariko commented 3 days ago

When attempting to create a new blueprint, I'm unable to proceed because I must specify the project name at that point. If I create the project first, the blueprint is automatically generated and attached to the project, preventing me from creating a blueprint specifically for that project afterward.

When trying to PATCH an existing blueprint, the Swagger API provided is:

curl -X 'PATCH' \
  'https://xxxxx/api/blueprints/{blueprint_id}' \
  -H 'accept: application/json'

However, this command does not allow me to specify which project to attach the blueprint to.

What is the purpose of the PATCH /blueprints/{blueprint_id} command? I need clarification on how to specify the project to which the blueprint should be attached.

jensrotne commented 3 days ago

@klesh I just did another test with the API using v1.0.0-rc1, and same issue on endpoint POST /api/projects

Example body:

{
  "name": "B2B3",
  "description": "",
  "metrics": [
    {
      "pluginName": "dora",
      "pluginOption": null,
      "enable": true
    }
  ],
  "blueprint": {
    "name": "B2B3-Blueprint",
    "projectName": "B2B3",
    "mode": "NORMAL",
    "plan": [
      "**REDACTED**"
    ],
    "enable": true,
    "cronConfig": "0 0 * * *",
    "isManual": false,
    "beforePlan": null,
    "afterPlan": null,
    "labels": [],
    "connections": [
      {
        "pluginName": "circleci",
        "connectionId": 2,
        "scopes": [
          {
            "scopeId": "1"
          },
          {
            "scopeId": "3"
          },
          {
            "scopeId": "4"
          }
        ]
      },
      {
        "pluginName": "github",
        "connectionId": 2,
        "scopes": [
          "**REDACTED**"
        ]
      }
    ],
    "skipOnFail": false,
    "skipCollectors": false,
    "fullSync": false
  }
}

It created the project fine, but the blueprint is missing the connections. I was still able to patch it afterwards using the PATCH /api/blueprints/{blueprint_id} endpoint.

jensrotne commented 3 days ago

There are no errors in the backend container either. Just got this log entry: lake time="2024-06-28 07:29:15" level=info msg="added blueprint 20 to cronjobs, cron id: 4, cron config: 0 0 * * *"

apache / incubator-devlake