fixed custom fields Double by adding number to the schema

[lpearson-trek10]

@visch I ran across an error recently that one of our custom fields had a double and none of the current values were able to parse that.

Error looked like:

time=2022-04-16 02:11:27 name=tap-clickup level=INFO message=INFO METRIC: {'type': 'timer', 'metric': 'http_request_duration', 'value': 14.763091, 'tags': {'endpoint': '/api/v2/team/8591733/task?include_closed=true&subtasks=true&page=63&archived=false&order_by=updated&reverse=true&date_updated_gt=0', 'http_status_code': 200, 'status': 'succeeded', 'context': {'archived': 'false', 'team_id': '8591733'}}}
time=2022-04-16 02:11:42 name=tap-clickup level=INFO message=INFO METRIC: {'type': 'timer', 'metric': 'http_request_duration', 'value': 14.662344, 'tags': {'endpoint': '/api/v2/team/8591733/task?include_closed=true&subtasks=true&page=64&archived=false&order_by=updated&reverse=true&date_updated_gt=0', 'http_status_code': 200, 'status': 'succeeded', 'context': {'archived': 'false', 'team_id': '8591733'}}}
time=2022-04-16 02:11:56 name=tap-clickup level=INFO message=INFO METRIC: {'type': 'timer', 'metric': 'http_request_duration', 'value': 14.241981, 'tags': {'endpoint': '/api/v2/team/8591733/task?include_closed=true&subtasks=true&page=65&archived=false&order_by=updated&reverse=true&date_updated_gt=0', 'http_status_code': 200, 'status': 'succeeded', 'context': {'archived': 'false', 'team_id': '8591733'}}}
time=2022-04-16 02:12:10 name=tap-clickup level=INFO message=INFO METRIC: {'type': 'timer', 'metric': 'http_request_duration', 'value': 14.309703, 'tags': {'endpoint': '/api/v2/team/8591733/task?include_closed=true&subtasks=true&page=66&archived=false&order_by=updated&reverse=true&date_updated_gt=0', 'http_status_code': 200, 'status': 'succeeded', 'context': {'archived': 'false', 'team_id': '8591733'}}}
time=2022-04-16 02:12:25 name=tap-clickup level=INFO message=INFO METRIC: {'type': 'timer', 'metric': 'http_request_duration', 'value': 14.927095, 'tags': {'endpoint': '/api/v2/team/8591733/task?include_closed=true&subtasks=true&page=67&archived=false&order_by=updated&reverse=true&date_updated_gt=0', 'http_status_code': 200, 'status': 'succeeded', 'context': {'archived': 'false', 'team_id': '8591733'}}}
time=2022-04-16 02:12:40 name=tap-clickup level=INFO message=INFO METRIC: {'type': 'timer', 'metric': 'http_request_duration', 'value': 14.606257, 'tags': {'endpoint': '/api/v2/team/8591733/task?include_closed=true&subtasks=true&page=68&archived=false&order_by=updated&reverse=true&date_updated_gt=0', 'http_status_code': 200, 'status': 'succeeded', 'context': {'archived': 'false', 'team_id': '8591733'}}}
time=2022-04-16 02:12:55 name=tap-clickup level=INFO message=INFO METRIC: {'type': 'timer', 'metric': 'http_request_duration', 'value': 15.438268, 'tags': {'endpoint': '/api/v2/team/8591733/task?include_closed=true&subtasks=true&page=69&archived=false&order_by=updated&reverse=true&date_updated_gt=0', 'http_status_code': 200, 'status': 'succeeded', 'context': {'archived': 'false', 'team_id': '8591733'}}}
INFO Serializing batch with 640 messages for table task
INFO Sending batch of 2072846 bytes to https://api.stitchdata.com/v2/import/batch
INFO Sending batch of 1950469 bytes to https://api.stitchdata.com/v2/import/batch
ERROR Giving up post_coroutine(...) after 1 tries (target_stitch.StitchClientResponseError)
INFO FLUSH early exit because of SEND_EXCEPTION: StitchClientResponseError()
CRITICAL Error persisting data to Stitch: 400: {'error': 'Record 294 for table task did not conform to schema:\n#/custom_fields: #: no subschema matched out of the total 2 subschemas\n#/custom_fields/3/value: #: no subschema matched out of the total 6 subschemas\n#/custom_fields/3/value: expected type: Integer, found: Double\n#/custom_fields/3/value: expected type: String, found: Double\n#/custom_fields/3/value: expected type: Boolean, found: Double\n#/custom_fields/3/value: expected: null, found: Double\n#/custom_fields/3/value: expected type: JSONArray, found: Double\n#/custom_fields/3/value: expected type: JSONObject, found: Double\n#/custom_fields: expected: null, found: JSONArray\n'}
[INFO]  2022-04-16T02:12:58.727Z    ed9c885f-5466-4499-9338-9aca16d33c5b    1
[INFO]  2022-04-16T02:13:13.459Z    ed9c885f-5466-4499-9338-9aca16d33c5b    No agent or invalid configuration file found

so I added number because that seemed to be he data type it was looking for http://json-schema.org/understanding-json-schema/reference/numeric.html#number also I reran it after changing this and it fixed it so seems like the right move.

[LucasZielke]

One of the task shows a new field that I think might be a cause of part of the error: '.tasks[1].custom_fields[0]'

{
  "id": "000000-0000-0000-0000-00000000",
  "name": "Process",
  "type": "drop_down",
  "type_config": {
    "new_drop_down": true,     <------- This is not included in either schemas
    "options": [
      {
        "id": "000000-0000-0000-0000-00000000",
        "name": "XXXXXXXXXXXX",
        "color": "#04A9F4",
        "orderindex": 0
      }
,,,

[visch]

Thank you! @LucasZielke / @lpearson-trek10 I'll dive into this when I can. It'll probably be early this coming week!

[visch]

One of the task shows a new field that I think might be a cause of part of the error: '.tasks[1].custom_fields[0]'

{
  "id": "000000-0000-0000-0000-00000000",
  "name": "Process",
  "type": "drop_down",
  "type_config": {
    "new_drop_down": true,     <------- This is not included in either schemas
    "options": [
      {
        "id": "000000-0000-0000-0000-00000000",
        "name": "XXXXXXXXXXXX",
        "color": "#04A9F4",
        "orderindex": 0
      }
,,,

@LucasZielke
I added the new drop down custom field to my clickup instance and didn't have the "new_drop_down": true, field come through. We can definitely add it if you'd like, right now it gets ignored (you'd see a WARNING print in stderr that we missed that field as well.