search

airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
16.32k stars 4.16k forks source link

Source Clickup: Tasks Filtered by Teams #26122

Open comer-tcr opened 1 year ago

comer-tcr commented 1 year ago

Connector Name

Clickup

Connector Version

0.1.1

What step the error happened?

During the sync

Revelant information

Source Name: ClickUp Destination Name: Postgres

It seens Clickup connector uses the endpoint (url = "https://api.clickup.com/api/v2/list/" + list_id + "/task") when fetching tasks dates, but this endpoint expect a list_id to work, not allowing to get tasks from several lists. There is another endpoint (url = "https://api.clickup.com/api/v2/team/" + team_id + "/task") that returns all taks from the specified team, making more sense to fetch all task and save it to a DW or DB. Besides that, the only paremeter that it´s possible to set is to include closed tasks, when there are lots of other parameters including being able to get subtasks: query = { "page": "0", "order_by": "string", "reverse": "true", "subtasks": "true", "space_ids": "string", "project_ids": "string", "list_ids": "string", "statuses": "string", "include_closed": "true", "assignees": "string", "tags": "string", "due_date_gt": "0", "due_date_lt": "0", "date_created_gt": "0", "date_created_lt": "0", "date_updated_gt": "0", "date_updated_lt": "0", "date_done_gt": "0", "date_done_lt": "0", "custom_fields": "string", "custom_task_ids": "true", "team_id": "123", "parent": "string" } I´m only getting a successfull conection when I fill all the fields in the source settings (folder_id, list_id, space_id and team_id) wich returns a very limited range of data that realy won´t aggregate much in temrns of analysis. 88ec12aa_ecc9_4c10_9ebc_96cb37ec4291_logs_90_txt.txt

Relevant log output

No response

Contribute

M00nNight commented 1 year ago

HI I love to work on it

aldrejota commented 1 year ago

@M00nNight happy to help here if you need assistance. Just finished the Pivotal Tracker (#30869) connector, also a Project Management app with similar data structure.

@justinbchau

justinbchau commented 1 year ago

@M00nNight Are you still actively working on this?

M00nNight commented 1 year ago

I'm currently working on it. I'm new to Python, which is why it's taking me some time.

justinbchau commented 1 year ago

Understood. Thank you for updating me

mrwilbroad commented 1 year ago

ooh i wish to participate in this but i'm beginner to python , maybe php-laravel , reactjs sometimes later

M00nNight commented 1 year ago

Yeah please do it 😊

mrwilbroad commented 1 year ago

Yeah please do it 😊 soon