airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
15.49k stars 3.99k forks source link

[source-freshdesk] Ticket Fields stream meta errors - Problem with choices #36699

Open jzamanxometry opened 5 months ago

jzamanxometry commented 5 months ago

Connector Name

freshdesk

Connector Version

3.0.7

What step the error happened?

During the sync

Relevant information

In the first sync of the ticket_fields stream, for status, group, cf_type, priority, agent, and source, the choices column wasn't populated and the error was Problem with choices. See screenshot below of what is in my snowflake database

Screenshot 2024-03-29 at 4 05 53 PM

Relevant log output

No response

Contribute

natikgadzhi commented 5 months ago

I gather you're using Airbyte OSS? We've just pushed up a big change in Freshdesk connector in 3.1.0 — can you run that one and see if the problem is resolved there?

jzamanxometry commented 5 months ago

Hi @natikgadzhi I don't see that version 3.1.0 available in my instance. Anything I can do to get it? Screenshot 2024-04-01 at 6 38 35 AM

jzamanxometry commented 5 months ago

Actually I just changed the version manually and it still had the same issue Screenshot 2024-04-01 at 6 54 14 AM

Screenshot 2024-04-01 at 6 54 54 AM

marcosmarxm commented 5 months ago

Not sure if this is because of how choices field is declared today: https://github.com/airbytehq/airbyte/blob/5e51b26663501779de9bda0166c77148cb683d13/airbyte-integrations/connectors/source-freshdesk/source_freshdesk/schemas/ticket_fields.json#L50-L52

@jzamanxometry would be possible to run a manual request and share how choices data is returning to your account?

jzamanxometry commented 5 months ago

@marcosmarxm Here's the JSON returned when I just query it via API

{
  "id": 70000471420,
  "name": "status",
  "label": "Status",
  "label_for_customers": "Status",
  "position": 3,
  "type": "default_status",
  "default": true,
  "customers_can_edit": false,
  "customers_can_filter": false,
  "required_for_closure": false,
  "required_for_agents": true,
  "required_for_customers": false,
  "displayed_to_customers": true,
  "created_at": "2021-07-13T17:28:35Z",
  "updated_at": "2023-02-16T16:25:08Z",
  "archived": false,
  "choices": [
    {
      "id": 2,
      "label": "Open",
      "label_for_customers": "Being Processed",
      "value": "Open",
      "stop_sla_timer": false,
      "default": true,
      "position": 1,
      "deleted": false
    },
    {
      "id": 3,
      "label": "Pending",
      "label_for_customers": "We're working on it!",
      "value": "Pending",
      "stop_sla_timer": true,
      "default": true,
      "position": 2,
      "deleted": false
    },
    {
      "id": 4,
      "label": "Resolved",
      "label_for_customers": "This ticket has been Resolved",
      "value": "Resolved",
      "stop_sla_timer": true,
      "default": true,
      "position": 3,
      "deleted": false
    },
    {
      "id": 5,
      "label": "Closed",
      "label_for_customers": "This ticket has been Closed",
      "value": "Closed",
      "stop_sla_timer": true,
      "default": true,
      "position": 4,
      "deleted": false
    },
    {
      "id": 6,
      "label": "Waiting on Requestor",
      "label_for_customers": "Awaiting your Reply",
      "value": "Waiting on Requestor",
      "stop_sla_timer": true,
      "default": false,
      "position": 5,
      "deleted": false,
      "group_ids": []
    },
    {
      "id": 7,
      "label": "Waiting on Third Party",
      "label_for_customers": "Reaching out to Vendor/Third Party",
      "value": "Waiting on Third Party",
      "stop_sla_timer": true,
      "default": false,
      "position": 6,
      "deleted": false,
      "group_ids": []
    },
    {
      "id": 8,
      "label": "DeactivateAccount",
      "label_for_customers": "Account Completely Deactivated",
      "value": "DeactivateAccount",
      "stop_sla_timer": true,
      "default": false,
      "position": 7,
      "deleted": false,
      "group_ids": []
    },
    {
      "id": 9,
      "label": "Waiting on Equipment Return",
      "label_for_customers": "Waiting on Equipment Return",
      "value": "Waiting on Equipment Return",
      "stop_sla_timer": true,
      "default": false,
      "position": 8,
      "deleted": false,
      "group_ids": []
    }
  ]
}

Screenshot 2024-04-10 at 1 43 45 PM