opensearch-project / sql

Query your data using familiar SQL or intuitive Piped Processing Language (PPL)
https://opensearch.org/docs/latest/search-plugins/sql/index/
Apache License 2.0
116 stars 134 forks source link

[BUG] NPE when create Spark data source #2496

Open noCharger opened 7 months ago

noCharger commented 7 months ago

What is the bug?

NPE when create Spark data source

How can one reproduce the bug?

curl --request POST \
  --url localhost:9200/_plugins/_query/_datasources \
  --header 'content-type: application/x-ndjson' \
  --data '{"name": "mys5","description": "","connector": "SPARK","allowedRoles": [],"properties": {"glue.auth.type": "iam_role","glue.auth.role_arn": "arn:aws:iam::123456:role/flint-opensearch-role","glue.indexstore.opensearch.uri": "http://localhost:9200","glue.indexstore.opensearch.auth": "noauth"}}'
{
  "status": 400,
  "error": {
    "type": "NullPointerException",
    "reason": "Invalid Request",
    "details": "Cannot invoke \"String.equals(Object)\" because the return value of \"java.util.Map.get(Object)\" is null"
  }
}%                            

## opensearch.log
Error happened during request handling
java.lang.NullPointerException: Cannot invoke "String.equals(Object)" because the return value of "java.util.Map.get(Object)" is null
        at org.opensearch.sql.spark.storage.SparkStorageFactory.getStorageEngine(SparkStorageFactory.java:78) ~[spark-3.0.0.0-SNAPSHOT.jar:?]
        at org.opensearch.sql.spark.storage.SparkStorageFactory.createDataSource(SparkStorageFactory.java:67) ~[spark-3.0.0.0-SNAPSHOT.jar:?]
        at org.opensearch.sql.datasources.service.DataSourceLoaderCacheImpl.getOrLoadDataSource(DataSourceLoaderCacheImpl.java:43) ~[datasources-3.0.0.0-SNAPSHOT.jar:?]
        at org.opensearch.sql.datasources.service.DataSourceServiceImpl.createDataSource(DataSourceServiceImpl.java:85) ~[datasources-3.0.0.0-SNAPSHOT.jar:?]
        at org.opensearch.sql.datasources.transport.TransportCreateDataSourceAction.doExecute(TransportCreateDataSourceAction.java:73) [datasources-3.0.0.0-SNAPSHOT.jar:?]
        at org.opensearch.sql.datasources.transport.TransportCreateDataSourceAction.doExecute(TransportCreateDataSourceAction.java:27) [datasources-3.0.0.0-SNAPSHOT.jar:?]
        at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:218) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.action.support.TransportAction.execute(TransportAction.java:188) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.action.support.TransportAction.execute(TransportAction.java:107) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.client.node.NodeClient.executeLocally(NodeClient.java:110) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.client.node.NodeClient.doExecute(NodeClient.java:97) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.client.support.AbstractClient.execute(AbstractClient.java:476) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.sql.datasources.rest.RestDataSourceQueryAction.lambda$executePostRequest$1(RestDataSourceQueryAction.java:145) [datasources-3.0.0.0-SNAPSHOT.jar:?]
        at org.opensearch.sql.datasources.utils.Scheduler.lambda$withCurrentContext$0(Scheduler.java:30) [datasources-3.0.0.0-SNAPSHOT.jar:?]
        at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:854) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) [?:?]
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) [?:?]
        at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]

What is the expected behavior? A clear and concise description of what you expected to happen.

If it's due to config missing, we should handle this gracefully.

What is your host/environment?

OS main sql main

noCharger commented 7 months ago
curl -XPOST localhost:9200/_plugins/_query/_datasources \
  --header 'content-type: application/x-ndjson' \
  --data '{"name": "mys5","description": "","connector": "SPARK","allowedRoles": [],"properties": {"glue.auth.type": "iam_role","glue.auth.role_arn": "arn:aws:iam::123456:role/flint-opensearch-role","glue.indexstore.opensearch.uri": "http://localhost:9200","glue.indexstore.opensearch.auth": "noauth"}}'   -u admin:admin --insecure
{
  "status": 400,
  "error": {
    "type": "NullPointerException",
    "reason": "Invalid Request",
    "details": "Cannot invoke \"String.equals(Object)\" because the return value of \"java.util.Map.get(Object)\" is null"
  }
}