AutoIDM / tap-clickup

tap-clickup , singer compliant tap for pulling clickup data
MIT License
12 stars 20 forks source link

Rename parent_id values and adding breadcrumbs #79

Open LucasZielke opened 3 years ago

LucasZielke commented 3 years ago

Feature Suggestion:

Having the parent_ids renamed relative to the attributed object, where a foldered_tasks parent_id then becomes list_id ect.

And adding breadcrumbs along the way for parent_ids of those parent_ids. These should be most of the relevant breadcrumbs:

visch commented 3 years ago

Right, so the context's that are passed from the Parent streams, those ID's are useful to analysis later on.

No reason not to add those I agree

https://github.com/AutoIDM/tap-clickup/blob/main/tap_clickup/streams.py#L100

visch commented 3 years ago

Part of me wonders if we should prefix the key with something, like _sdc_ to indicate that this data isn't from the API response

visch commented 2 years ago

2022-04-29T01:39:52.181133Z [info ] time=2022-04-28 21:39:52 name=tap-clickup level=WARNING message=Property 'space_id' was present in the 'folder' stream but not found in catalog schema. Ignoring. cmd_type=extractor job_id=2022-04-29T013944--tap-clickup--target-jsonl name=tap-clickup run_id=5f4de51e-8f1b-4519-a6f4-0db74b546195 stdio=stderr 2022-04-29T01:39:52.181551Z [info ] time=2022-04-28 21:39:52 name=tap-clickup level=WARNING message=Property 'team_id' was present in the 'space' stream but not found in catalog schema. Ignoring. cmd_type=extractor job_id=2022-04-29T013944--tap-clickup--target-jsonl name=tap-clickup run_id=5f4de51e-8f1b-4519-a6f4-0db74b546195 stdio=stderr