When dealing with enterprise workspaces in ClickUp, as ClickUp admins, we often find ourselves having access to multiple workspaces, lists, tasks, and other elements that are not needed in our pipeline. To save resources, processing time, and improve efficiency, it is beneficial for us to have the ability to filter specific elements that we want to fetch.
Noteworthy ClickUp API Bugs
During the development of this project, I encountered some inherent bugs in ClickUp's API. Primarily, these bugs prevented the passing of a single space or list as a parameter. However, a workaround was discovered: by passing the same ID twice, the API would function correctly, and there are not any duplicate values in the API response.
Issues
The following changes have been implemented and work, but they are currently coded to only fetch data from the general settings in the meltano.yml file. Consequently, it is not possible to configure different pipelines for various environments because they are not being correctly read.
For example, when using the command meltano --environment=dev run tap-clickup ..., the configurations are not being applied as expected from the dev envv because it reads only from the common configurations.
Taking this meltano.yml sample, the code always reads "spaces_id" as '90100424497' even when we run it in prod (which in this case should be '90070255202'
Could not figure out how to correctly add the new setting in the meltano.yml file. At first I was just hardcoding it to the file, then I found out about tap-clickup--autoidm.lock file and in this file I added the new configuration settings to be added when using meltano config tap-clickup -set. But I cannot figure out how to edit this into the projects code, i basically added these configs to the og file:
. . .
{
"name": "stream_map_config",
"kind": "object",
"label": "Stream Map Config",
"description": "User-defined config values to be used within map expressions."
},
{
"name": "stream_maps",
"kind": "object",
"label": "Stream Maps",
"description": "Config object for stream maps capability. For more information check out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html)."
},
{
"name": "workspace_id",
"kind": "integer",
"label": "Workspace Id",
"description": "To limit from which workspace to fetch data pass it's id(https://dev-doc.clickup.com/333/d/h/ad-455845/c464293970a6906)."
},
{
"name": "spaces_id",
"kind": "string",
"label": "Spaces Ids",
"description": "To limit from which Spaces to fetch data from, pass their ids as a list string Example: spaceid1,spaceid2,ispaced3."
},
{
"name": "list_ids",
"kind": "string",
"label": "Lists Ids",
"description": "To limit from which Lists to fetch data from, pass their ids as a list string Example: listid1,listid2,listid3."
}
]
}
Description
When dealing with enterprise workspaces in ClickUp, as ClickUp admins, we often find ourselves having access to multiple workspaces, lists, tasks, and other elements that are not needed in our pipeline. To save resources, processing time, and improve efficiency, it is beneficial for us to have the ability to filter specific elements that we want to fetch.
Noteworthy ClickUp API Bugs
During the development of this project, I encountered some inherent bugs in ClickUp's API. Primarily, these bugs prevented the passing of a single space or list as a parameter. However, a workaround was discovered: by passing the same ID twice, the API would function correctly, and there are not any duplicate values in the API response.
Issues
For example, when using the command meltano --environment=dev run tap-clickup ..., the configurations are not being applied as expected from the dev envv because it reads only from the common configurations.
Taking this meltano.yml sample, the code always reads "spaces_id" as '90100424497' even when we run it in prod (which in this case should be '90070255202'