apache / drill

Apache Drill is a distributed MPP query layer for self describing data
https://drill.apache.org/
Apache License 2.0
1.93k stars 979 forks source link

it cause an exception when query http plugin data with fullName #2851

Open shfshihuafeng opened 10 months ago

shfshihuafeng commented 10 months ago

Before submitting a bug report, please verify that you are using the most current version of Drill.

Describe the bug i create a http plugin named "http" and then use "use http" command to switch schema. i query http Data with "select * from http.sunrise",It report except

To Reproduce Steps to reproduce the behavior:

1. create plugin with name "http"
2. use http
3. select * from http.sunrise;

Expected behavior

apache drill> select * from http.sunrise;
+----------------------------------------------------------------------------------+--------+
|                                     results                                      | status |
+----------------------------------------------------------------------------------+--------+
| {"sunrise":"5:42:47 AM","sunset":"5:52:15 PM","solar_noon":"11:47:31 AM","day_length":"12:09:28","civil_twilight_begin":"5:21:47 AM","civil_twilight_end":"6:13:15 PM","nautical_twilight_begin":"4:56:00 AM","nautical_twilight_end":"6:39:02 PM","astronomical_twilight_begin":"4:30:08 AM","astronomical_twilight_end":"7:04:54 PM"} | OK     |
+----------------------------------------------------------------------------------+--------+
1 row selected (2.967 seconds)

Error detail, log output or screenshots Error: CONNECTION ERROR: API 'http' does not exist in HTTP storage plugin 'http'

Drill version 1.22.0

Additional context

http plugin config
{
  "type": "http",
  "connections": {
    "sunrise": {
      "url": "https://api.sunrise-sunset.org/json",
      "requireTail": false,
      "method": "GET",
      "params": [
        "lat",
        "lng",
        "date"
      ],
      "authType": "none",
      "inputType": "json",
      "xmlDataLevel": 1,
      "postParameterLocation": "QUERY_STRING",
      "verifySSLCert": true
    }
  },
  "timeout": 5,
  "retryDelay": 1000,
  "proxyType": "direct",
  "authMode": "SHARED_USER",
  "enabled": true
}
shfshihuafeng commented 10 months ago

we should determines whether the parameter name equals the plug-in name ?

--- a/contrib/storage-http/src/main/java/org/apache/drill/exec/store/http/HttpSchemaFactory.java
+++ b/contrib/storage-http/src/main/java/org/apache/drill/exec/store/http/HttpSchemaFactory.java
@@ -18,8 +18,11 @@
 package org.apache.drill.exec.store.http;

 import java.util.Collections;
+import java.util.Locale;
 import java.util.Map;
 import java.util.Map.Entry;
+import java.util.Set;
+import java.util.stream.Collectors;

 import org.apache.calcite.schema.SchemaPlus;
 import org.apache.calcite.schema.Table;
@@ -85,7 +88,7 @@ public class HttpSchemaFactory extends AbstractSchemaFactory {
       HttpAPIConnectionSchema subSchema = subSchemas.get(name);
       if (subSchema != null) {
         return subSchema;
-      } else if (tables.containsKey(name)) {
+      } else if (tables.containsKey(name) || isRegistryPluginName(name.toLowerCase(Locale.ROOT))) {
         return null;
       } else {
         throw UserException
@@ -95,6 +98,12 @@ public class HttpSchemaFactory extends AbstractSchemaFactory {
       }
     }

+    private boolean isRegistryPluginName(String name) {
+      Set<String> pluginNames = plugin.getRegistry().availablePlugins();
+      Set<String> pluginNamesToLower = pluginNames.stream().map(String::toLowerCase).collect(Collectors.toSet());
+      return pluginNamesToLower.contains(name);
+    }
cgivre commented 10 months ago

@shfshihuafeng I don't think this is a bug. You are first running the query USE http which sets the root at http. Then you're running a SELECT ... FROM http.sunrise. So at that point, the plugin is looking for a path: http.http.sunrise which does not exist.

I'd bet that if you either skipped the USE query or ran a SELECT ... FROM sunrise it would work.

Javelin2007 commented 10 months ago

@cgivre You are right, but when use http, the default schema is http, when we use another data source schema, will report error. image It think this is not right.

shfshihuafeng commented 10 months ago

@cgivre image

shfshihuafeng commented 10 months ago

@cgivre when i use "use my.test" command to switch schema, It is correct to query data with full schema Name from other data sources 0880c4c80710fc8ae113

cgivre commented 10 months ago

@shfshihuafeng So are you saying the behavior with USE is inconsistent? I want to make sure I understand the issue.

shfshihuafeng commented 10 months ago

@cgivre There are two problems with this

  1. behavior with USE is inconsistent
  2. when i enter the scehma with 'use',data can not be queried image