iamazy / elasticsearch-sql

parse sql into elasticsearch dsl with antlr4
https://iamazy.github.io/elasticsearch-sql/
MIT License
339 stars 89 forks source link

JDBC not support 'describe'. #97

Closed wellCh4n closed 1 year ago

wellCh4n commented 1 year ago

Describe expression could be parsed by ElasticSql2DslParser. But SqlOperation.DESC is assert false in executeQuery.

I implemented an example for this problem.

ElasticSqlParseResult parse = PARSER.parse(statement);
org.elasticsearch.action.admin.indices.mapping.get.GetMappingsRequest parseRequest = parse.getMappingsRequest();

// Converting requests, as the old GetMappingsRequest implementation is deprecated in RestHighLevelClient
GetMappingsRequest request = new GetMappingsRequest();
request.indices(parseRequest.indices());

GetMappingsResponse mapping = elasticConnection.getRestClient()
        .indices()
        .getMapping(request, RequestOptions.DEFAULT);

But GetMappingsResponse is a complex object, converting to a result set is not so easy because ElasticSearch Mapping is not a two-dimensional table. Such as

"name": {
    "type": "text",
    "fields": {
        "keyword": {
            "type": "keyword",
            "ignore_above": 256
        }
    }
}

We can convert type information to JSON String, in my opinion. But this does not look very native. Maybe we need to discuss the situation. LFTYR! In the end, I love this project very much!

wellCh4n commented 1 year ago

And... If we use 'assert' to throw exception, we need to add '-ea' to vm options, otherwise the DESC operation will still go to the SELECT logic ~

iamazy commented 1 year ago

Describe expression could be parsed by ElasticSql2DslParser. But SqlOperation.DESC is assert false in executeQuery.

I implemented an example for this problem.

ElasticSqlParseResult parse = PARSER.parse(statement);
org.elasticsearch.action.admin.indices.mapping.get.GetMappingsRequest parseRequest = parse.getMappingsRequest();

// Converting requests, as the old GetMappingsRequest implementation is deprecated in RestHighLevelClient
GetMappingsRequest request = new GetMappingsRequest();
request.indices(parseRequest.indices());

GetMappingsResponse mapping = elasticConnection.getRestClient()
        .indices()
        .getMapping(request, RequestOptions.DEFAULT);

But GetMappingsResponse is a complex object, converting to a result set is not so easy because ElasticSearch Mapping is not a two-dimensional table. Such as

"name": {
    "type": "text",
    "fields": {
        "keyword": {
            "type": "keyword",
            "ignore_above": 256
        }
    }
}

We can convert type information to JSON String, in my opinion. But this does not look very native. Maybe we need to discuss the situation. LFTYR! In the end, I love this project very much!

@wellCh4n , Thanks! What about flatten JSON into a two-dimensional table, e.g.

from

"name": {
    "type": "text",
    "fields": {
        "keyword": {
            "type": "keyword",
            "ignore_above": 256
        }
    }
}

to

{
   "name.type": "text",
   "name.fields.keyword.type": "keyword",
   "name.fields.keyword.ignore_above": 256
}
wellCh4n commented 1 year ago

Describe expression could be parsed by ElasticSql2DslParser. But SqlOperation.DESC is assert false in executeQuery. I implemented an example for this problem.

ElasticSqlParseResult parse = PARSER.parse(statement);
org.elasticsearch.action.admin.indices.mapping.get.GetMappingsRequest parseRequest = parse.getMappingsRequest();

// Converting requests, as the old GetMappingsRequest implementation is deprecated in RestHighLevelClient
GetMappingsRequest request = new GetMappingsRequest();
request.indices(parseRequest.indices());

GetMappingsResponse mapping = elasticConnection.getRestClient()
        .indices()
        .getMapping(request, RequestOptions.DEFAULT);

But GetMappingsResponse is a complex object, converting to a result set is not so easy because ElasticSearch Mapping is not a two-dimensional table. Such as

"name": {
    "type": "text",
    "fields": {
        "keyword": {
            "type": "keyword",
            "ignore_above": 256
        }
    }
}

We can convert type information to JSON String, in my opinion. But this does not look very native. Maybe we need to discuss the situation. LFTYR! In the end, I love this project very much!

@wellCh4n , Thanks! What about flatten JSON into a two-dimensional table, e.g.

from

"name": {
    "type": "text",
    "fields": {
        "keyword": {
            "type": "keyword",
            "ignore_above": 256
        }
    }
}

to

{
   "name.type": "text",
   "name.fields.keyword.type": "keyword",
   "name.fields.keyword.ignore_above": 256
}

@iamazy Thanks for your reply. Flatten is a good idea! But the number of columns in the ResultSet becomes uncertain. For MySQL, the result set described is a two-dimensional table with a determined number of rows.

image

If we flatten the result, the result will be

image

I tried ClickHouse and the results are not the same as MySQL. image

I understand that different database types, people are able to accept desc's return differently. So I tend to convert the other fields to JSON String. In my opinion, this is the most intuitive. e.g.

image

info is just a temporary name

Looking forward to your suggestions! 😆

wellCh4n commented 1 year ago

But ElasticSearch is a document(JSON)-based database. Is there no need to think about other databases, the original mapping as a return is ok?

iamazy commented 1 year ago

I understand that different database types, people are able to accept desc's return differently. So I tend to convert the other fields to JSON String. In my opinion, this is the most intuitive. e.g.

image

I think this way is a better choice, do you willing to submit a pr?

wellCh4n commented 1 year ago

I understand that different database types, people are able to accept desc's return differently. So I tend to convert the other fields to JSON String. In my opinion, this is the most intuitive. e.g.

image

I think this way is a better choice, do you willing to submit a pr?

That's great! I will commit a PR this weekend~ And, how about changing 'info' to 'extra' in reference to MySQL?

iamazy commented 1 year ago

I understand that different database types, people are able to accept desc's return differently. So I tend to convert the other fields to JSON String. In my opinion, this is the most intuitive. e.g.

image

I think this way is a better choice, do you willing to submit a pr?

That's great! I will commit a PR this weekend~ And, how about changing 'info' to 'extra' in reference to MySQL?

That's OK