search

airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
15.88k stars 4.07k forks source link

Source ElasticSearch - cannot read match_only_text field type, and causes a critical error #21479

Open AlexAVL opened 1 year ago

AlexAVL commented 1 year ago
## Environment - **Airbyte version**: 0.40.28 - **OS Version / Instance**: AWS EC2 - **Deployment**: Docker - **Source Connector and version**: Elasticsearch - **Destination Connector and version**: Redshift - **Step where error happened**: Setup new connection ## Current Behavior The connection fails with "Failed to fetch schema. Please try again" and "Error: common.error". It probably needs this file type added to the type mapper as a string. ## Expected Behavior The schema should be read correctly. ## Logs ``` Stack Trace: java.lang.RuntimeException: io.airbyte.integrations.source.elasticsearch.UnsupportedDatatypeException: Cannot map unsupported data type to Airbyte data type: match_only_text at io.airbyte.integrations.source.elasticsearch.typemapper.ElasticsearchTypeMapper.lambda$formatJSONSchema$0(ElasticsearchTypeMapper.java:133) at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133) at io.airbyte.integrations.source.elasticsearch.typemapper.ElasticsearchTypeMapper.formatJSONSchema(ElasticsearchTypeMapper.java:129) at io.airbyte.integrations.source.elasticsearch.typemapper.ElasticsearchTypeMapper.lambda$formatJSONSchema$0(ElasticsearchTypeMapper.java:131) at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133) at io.airbyte.integrations.source.elasticsearch.typemapper.ElasticsearchTypeMapper.formatJSONSchema(ElasticsearchTypeMapper.java:129) ``` ## Steps to Reproduce 1. Attempt to sync the schemas for an Elasticsearch deployment with an index which contains a `match_only_text` field type. ## Are you willing to submit a PR? No
octavia-squidington-iii commented 5 months ago

At Airbyte, we seek to be clear about the project priorities and roadmap. This issue has not had any activity for 180 days, suggesting that it's not as critical as others. It's possible it has already been fixed. It is being marked as stale and will be closed in 20 days if there is no activity. To keep it open, please comment to let us know why it is important to you and if it is still reproducible on recent versions of Airbyte.

octavia-squidington-iii commented 4 months ago

This issue was closed because it has been inactive for 20 days since being marked as stale.