Closed visch closed 2 years ago
Probably is a bug on target-bigquery's end but it's still curious and I'd like to be sure it's not a clickup thing.
Thanks to some additional debugging info added to target-bigquery, see https://github.com/adswerve/target-bigquery/pull/27, I can pin-point the problems to the folder schema in tap-clickup not matching the emitted records.
INFO Cannot format a record for stream folder to its corresponding BigQuery schema. Details: {'record': {'id': '12933951', 'name': 'Milestone and Project Plan', 'orderindex': 17, 'override_statuses': False, 'hidden': False, 'space': {'id': '2577684', 'name': 'meshDelivery'}, 'task_count': '10', 'archived': True, 'statuses': [], 'lists': [{'id': '25670974', 'name': 'POC <customer/department>', 'orderindex': 0, 'status': None, 'priority': None, 'assignee': None, 'task_count': 10, 'due_date': None, 'start_date': None, 'space': {'id': '2577684', 'name': 'meshDelivery', 'access': True}, 'archived': False, 'override_statuses': None, 'statuses': [{'id': 'p2577684_eDZ87cTk', 'status': 'Open', 'orderindex': 0, 'color': '#d3d3d3', 'type': 'open'}, {'id': 'p2577684_Sf8kB74x', 'status': 'planned', 'orderindex': 1, 'color': '#82CB11', 'type': 'custom'}, {'id': 'p2577684_yG5b2doG', 'status': 'in progress', 'orderindex': 2, 'color': '#4194f6', 'type': 'custom'}, {'id': 'p2577684_BZKpph7f', 'status': 'review', 'orderindex': 3, 'color': '#A875FF', 'type': 'custom'}, {'id': 'p2577684_ouoISXPV', 'status': 'Closed', 'orderindex': 4, 'color': '#6bc950', 'type': 'closed'}], 'permission_level': 'create'}], 'permission_level': 'create'}, 'schema': {'properties': {'id': {'type': ['string']}, 'name': {'type': ['null', 'string']}, 'orderindex': {'type': ['null', 'integer']}, 'override_statuses': {'type': ['null', 'boolean']}, 'hidden': {'type': ['null', 'boolean']}, 'space': {'properties': {'id': {'type': ['null', 'string']}, 'name': {'type': ['null', 'string']}}, 'type': 'object'}, 'task_count': {'type': ['null', 'string', 'integer']}, 'statuses': {'items': {}, 'type': ['array', 'null']}, 'lists': {'items': {}, 'type': ['array', 'null']}, 'archived': {'type': ['null', 'boolean']}, 'permission_level': {'type': ['null', 'string']}}, 'type': 'object'}, 'bq_schema': {'id': {'type': 'STRING', 'mode': 'REQUIRED', 'policyTags': {'names': []}}, 'name': {'type': 'STRING', 'mode': 'NULLABLE', 'policyTags': {'names': []}}, 'orderindex': {'type': 'INTEGER', 'mode': 'NULLABLE', 'policyTags': {'names': []}}, 'override_statuses': {'type': 'BOOLEAN', 'mode': 'NULLABLE', 'policyTags': {'names': []}}, 'hidden': {'type': 'BOOLEAN', 'mode': 'NULLABLE', 'policyTags': {'names': []}}, 'space': {'type': 'RECORD', 'mode': 'NULLABLE', 'fields': {'id': {'type': 'STRING', 'mode': 'NULLABLE', 'policyTags': {'names': []}}, 'name': {'type': 'STRING', 'mode': 'NULLABLE', 'policyTags': {'names': []}}}}, 'task_count': {'type': 'STRING', 'mode': 'NULLABLE', 'policyTags': {'names': []}}, 'statuses': {'type': 'RECORD', 'mode': 'REPEATED', 'fields': []}, 'lists': {'type': 'RECORD', 'mode': 'REPEATED', 'fields': []}, 'archived': {'type': 'BOOLEAN', 'mode': 'NULLABLE', 'policyTags': {'names': []}}, 'permission_level': {'type': 'STRING', 'mode': 'NULLABLE', 'policyTags': {'names': []}}, '_time_extracted': {'type': 'timestamp', 'mode': 'NULLABLE', 'policyTags': {'names': []}}, '_time_loaded': {'type': 'timestamp', 'mode': 'NULLABLE', 'policyTags': {'names': []}}}} cmd_type=loader job_id=clickup name=target-bigquery-truncate run_id=2b24a283-f8d0-44b6-be73-779fa86f68cc stdio=stderr
Also affected
all of these fields are declared as "type": ["array", "null"]
and with no schema for items
.
I believe this issue was fixed/handled on the target's side recently in https://github.com/adswerve/target-bigquery/pull/33.
Thank you @woochica , I'll close this
From @JohannesRudolph