Handle archived tasks and tasks in archived lists

[JohannesRudolph]

The ClickUp Tasks API endpoint has a couple of issues that are hard to work around. I see that https://github.com/AutoIDM/tap-clickup/blob/1115eafa2f3ddbb9890895fec6fa649f4940a612/tap_clickup/streams.py#L184 already handles fetching archived tasks (via a partition). This is good.

However what we're still missing is a way to get tasks form archived lists, as clickup does not include those. Tasks and Lists have their own "archived" flag, and a list with list.archived=true can have tasks with task.archived=false, yet the task is still not returned from the "Tasks / Get Filtered Team Tasks" API.

To my knowledge there's no good alternative (other than ClickUp fixing their API game...), other than recursing through the space -> folder hierarchy to find all archived lists, then loading tasks for each of those lists explicitly.

At the moment this means that tap_clickup may not fetch all tasks.

[visch]

The idea with making the Parent for Tasks be Teams is that all tasks from a Team would come through (including archived tasks). You're saying right now archived tasks are not coming through (it should be? ).

We actually started down that path first, and then we had complaints that merging together all the task streams was a pain due to tasks being deleted from one and moving to another (when archived and things)

[JohannesRudolph]

The problem here is the Clickup API. The tap is currently doing the right thing by fetching tasks with archived=true and archived false.

However it's omitting tasks (with either task.archived=true or task.archived=false) that are on a list.archived=true. So

• ✅ task.archived= false in list.archived = false
• ✅ task.archived= true in list.archived = false
• 🚫 task.archived= false in list.archived = true
• 🚫 task.archived= true in list.archived = true

This is not the fault of the tap, but rather the ClickUp API "tasks by team" endpoint not returning tasks from archived lists at all. At least last time I checked this! The only workaround would be to query archived lists separately.

[JohannesRudolph]

Oh, I now see you may have already described this same issue in https://github.com/AutoIDM/tap-clickup/issues/87

[visch]

@JohannesRudolph I went ahead and close that issue as it was old (also cleaned up the rest of the tasks, they should be good to go now)

Thanks again for submitting this. Next steps I think are

1. Create an archived list in clickup with a task, verify the task isn't brought over
2. Ask clickup support about the issue to see if they know about this / can fix it (probably not but worth a shot)
3. Decide on adding a new stream, or dealing with this in the current Tasks Stream

[visch]

@JohannesRudolph Thanks again, really appreciate the detail! I hope this tap is working for you!