AutoIDM / tap-clickup

tap-clickup , singer compliant tap for pulling clickup data
MIT License
12 stars 19 forks source link

[bug] Tasks stream does not return "labels" custom field names #132

Closed balmasi closed 1 year ago

balmasi commented 2 years ago

According to the clickup docs, the "Label" custom field type returns the name associated with a particular label in the label field of the type_config.options object array.

e.g.

{
    "name": "My Label Field",
    "type": "labels",
    "type_config": {
        "options": [
            {
                "id": "UUID",
                "label": "Label 1",
                "color": "#123456"
            },
            {
                "id": "UUID",
                "label": "Label 2",
                "color": "#FFFFFF"
            }
        ]
    }
}

Unfortunately, this label field is not included in the task stream's schema, so it's missing from the data stream output.

Current schema for the task stream at the custom_fields[].type_config.options[] level is:

{
  "options": {
    "type": ["array", "null"],
    "items": {
        "type": "object",
        "properties": {
            "id": {
                "type": ["string", "null"]
            },
            "name": {
                "type": ["string", "null"]
            },
            "color": {
                "type": ["string", "null"]
            },
            "orderindex": {
                "type": ["integer", "string", "null"]
            }
        }
    }
  }
}

By changing the schema to the one below, the implementation will start working correctly.

{
  "options": {
    "type": ["array", "null"],
    "items": {
        "type": "object",
        "properties": {
            "id": {
                "type": ["string", "null"]
            },
            "name": {
                "type": ["string", "null"]
            },
            "label": {
                "type": ["string", "null"]
            },
            "color": {
                "type": ["string", "null"]
            },
            "orderindex": {
                "type": ["integer", "string", "null"]
            }
        }
    }
  }
}

To reproduce this issue, simply create a task with a custom labels field and extract the data via target-jsonl. Notice that the label field is missing and there's no way to resolve the ids in the value field of the custom field to actual label names.

I've tried adding the field to the schema and can confirm that this resolves the issue.

It's worth mentioning that simply adding the field in the singer catalog file also resolves the issue

visch commented 2 years ago

According to the clickup docs, the "Label" custom field type returns the name associated with a particular label in the label field of the type_config.options object array.

e.g.

{
    "name": "My Label Field",
    "type": "labels",
    "type_config": {
        "options": [
            {
                "id": "UUID",
                "label": "Label 1",
                "color": "#123456"
            },
            {
                "id": "UUID",
                "label": "Label 2",
                "color": "#FFFFFF"
            }
        ]
    }
}

Unfortunately, this label field is not included in the task stream's schema, so it's missing from the data stream output.

Current schema for the task stream at the custom_fields[].type_config.options[] level is:

{
  "options": {
    "type": ["array", "null"],
    "items": {
        "type": "object",
        "properties": {
            "id": {
                "type": ["string", "null"]
            },
            "name": {
                "type": ["string", "null"]
            },
            "color": {
                "type": ["string", "null"]
            },
            "orderindex": {
                "type": ["integer", "string", "null"]
            }
        }
    }
  }
}

By changing the schema to the one below, the implementation will start working correctly.

{
  "options": {
    "type": ["array", "null"],
    "items": {
        "type": "object",
        "properties": {
            "id": {
                "type": ["string", "null"]
            },
            "name": {
                "type": ["string", "null"]
            },
            "label": {
                "type": ["string", "null"]
            },
            "color": {
                "type": ["string", "null"]
            },
            "orderindex": {
                "type": ["integer", "string", "null"]
            }
        }
    }
  }
}

To reproduce this issue, simply create a task with a custom labels field and extract the data via target-jsonl. Notice that the label field is missing and there's no way to resolve the ids in the value field of the custom field to actual label names.

I've tried adding the field to the schema and can confirm that this resolves the issue.

It's worth mentioning that simply adding the field in the singer catalog file also resolves the issue

Awesome, looks like we missed one (Maybe the API changed from when we wrote this or maybe we didn't have labels when we generated it) regardless this changes looks harmless.

If you could submit a PR with that change the file you're looking for that has that is here https://github.com/AutoIDM/tap-clickup/blob/main/tap_clickup/schemas/task.json#L195

If you submit that change I'll get it in right away!

Thanks again!

visch commented 1 year ago

@balmasi completely forgot about this and just touched clickup today so I put in this change here https://github.com/AutoIDM/tap-clickup/pull/139 Merging shortly, and we'll get a new release out to pypi soon as well