StarRocks / starrocks

The world's fastest open query engine for sub-second analytics both on and off the data lakehouse. With the flexibility to support nearly any scenario, StarRocks provides best-in-class performance for multi-dimensional analytics, real-time analytics, and ad-hoc queries. A Linux Foundation project.
https://starrocks.io
Apache License 2.0
8.89k stars 1.78k forks source link

[BUG] Elasticsearch category table does not exist when the es index contains dynamic_templates in the mapping #44536

Open jdzchenxin opened 6 months ago

jdzchenxin commented 6 months ago

Steps to reproduce the behavior (Required)

  1. Create elasticsearch index "test_index"
    PUT test_index
    {
    "mappings": {
    "dynamic_templates": [
       {
        "tags_template": {
          "match_mapping_type": "string",
          "match":   "tags*",
          "mapping": {
            "type": "keyword"
          }
        }
      }
    ],
    "properties": {
      "id": {
        "type": "keyword"
      },
      "title": {
        "type": "text"
      }
    }
    }
    }
  2. Create elasticsearch catalog in starrocks
    CREATE EXTERNAL CATALOG es_catalog
    comment "ES CATALOG"
    PROPERTIES (
    "hosts"  =  "https://127.0.0.1:9200",
    "type"  =  "es",
    "user"  =  "xxxxxx",
    "password" = "xxxxxx",
    "es.nodes.wan.only" = "true"
    )
  3. Verify table and select data in starrocks
    mysql> SET CATALOG es8_catalog;
    mysql> USE default_db;
    mysql> show tables like 'test_index%';
    mysql> select * from test_index;

Expected behavior (Required)

  1. show table return 1 result
  2. "select * from test_index" return success;

Real behavior (Required)

  1. show table return 0 result
  2. "select * from test_index" return :"ERROR 1064 (HY000): Getting analyzing error. Detail message: Unknown table 'default_db.test_index'."

Later, I drop the es_catalog, and rebuild it without part of "dynamic_templates", the table appeared in show tables list, and i success select it with the SQL.

StarRocks version (Required)

4.2.4-613f0b5

Elasticsearch version

8.8.2

github-actions[bot] commented 1 week ago

We have marked this issue as stale because it has been inactive for 6 months. If this issue is still relevant, removing the stale label or adding a comment will keep it active. Otherwise, we'll close it in 10 days to keep the issue queue tidy. Thank you for your contribution to StarRocks!

jmorcar commented 1 day ago

Same problem starrocks v3.3.4 , I think documentation is not complete about elasticsearch external catalog. Only mention function esquery() when is an external table function (deprecated) in starrocks.

image

Or maybe is mysql client not compatible with table names of external catalog?