Status code '405 Method Not Allowed' was returned from ElasticSearchStore

[jiangpengcheng]

When I use ElasticSearchStore to fetch activation logs, system will report such error, I think it is raised from elasticsearch's _search API

Then I checked the code, and found below in whisk/core/containerpool/logging/ElasticSearchRestClient.scala:

  def search[T: RootJsonReader](index: String,
                                payload: EsQuery = EsQuery(EsQueryAll()),
                                headers: List[HttpHeader] = List.empty): Future[Either[StatusCode, T]] =
    requestJson[T](mkJsonRequest(POST, Uri(index), payload.toJson.asJsObject, baseHeaders ++ headers))

what confused me is that the uri for search in ES should be like ${index_name}/_search while uri in above codes is not

I'm not sure whether this is a bug or meant to be like this(then I must have missed something)

Environment details:

• lambda version: latest master branch
• elasticsearch version: 6.3.2

Steps to reproduce the issue:

1. start a elasticsearch container

docker run -td --name es -p 9200:9200 docker.elastic.co/elasticsearch/elasticsearch:6.3.2

2. edit config for logstash and start it

input {
  file {
    path => "/tmp/wsklogs/invoker0/userlogs-*.log"
    codec => "json"
  }
}

output {
  elasticsearch{
    hosts => ["http://x.x.x.x:9200"]
    index => "whisk_user_logs"
  }
}

3. deploy the openwhisk on a single node, with below options added to ansible/group_vars/all

userLogs_spi: whisk.core.containerpool.logging.ElasticSearchLogStoreProvider
invoker_extraEnv:
  - CONFIG_whisk_logstore_elasticsearch_protocol: http
  - CONFIG_whisk_logstore_elasticsearch_host: x.x.x.x
  - CONFIG_whisk_logstore_elasticsearch_port: 9200
  - CONFIG_whisk_logstore_elasticsearch_path: /whisk_user_logs
  - CONFIG_whisk_logstore_elasticsearch_logSchema_userLogs: logs
  - CONFIG_whisk_logstore_elasticsearch_logSchema_message: namespaceId
  - CONFIG_whisk_logstore_elasticsearch_logSchema_activationId: activationId
  - CONFIG_whisk_logstore_elasticsearch_logSchema_stream: stream
  - CONFIG_whisk_logstore_elasticsearch_logSchema_time: time
controller_extraEnv:
  - CONFIG_whisk_logstore_elasticsearch_protocol: http
  - CONFIG_whisk_logstore_elasticsearch_host: x.x.x.x
  - CONFIG_whisk_logstore_elasticsearch_port: 9200
  - CONFIG_whisk_logstore_elasticsearch_path: /whisk_user_logs
  - CONFIG_whisk_logstore_elasticsearch_logSchema_userLogs: logs
  - CONFIG_whisk_logstore_elasticsearch_logSchema_message: namespaceId
  - CONFIG_whisk_logstore_elasticsearch_logSchema_activationId: activationId
  - CONFIG_whisk_logstore_elasticsearch_logSchema_stream: stream
  - CONFIG_whisk_logstore_elasticsearch_logSchema_time: time

4. invoke an action and get its activation's log:

curl -k https://789c46b1-71f6-4ed5-8c54-816aa4f8c502:abczO3xZCLrMN6v2BKK1dXYFpXlPkccOFqm12CdAsMgRU4VrNZ9lyGVCGuMDGIwP@localhost/api/v1/namespaces/_/activations/38598e4c588f43f6998e4c588f53f666/logs

Provide the expected results and outputs:

# curl -k https://789c46b1-71f6-4ed5-8c54-816aa4f8c502:abczO3xZCLrMN6v2BKK1dXYFpXlPkccOFqm12CdAsMgRU4VrNZ9lyGVCGuMDGIwP@localhost/api/v1/namespaces/_/activations/38598e4c588f43f6998e4c588f53f666/logs
{"logs":[]}

Provide the actual results and outputs:

# curl -k https://789c46b1-71f6-4ed5-8c54-816aa4f8c502:abczO3xZCLrMN6v2BKK1dXYFpXlPkccOFqm12CdAsMgRU4VrNZ9lyGVCGuMDGIwP@localhost/api/v1/namespaces/_/activations/38598e4c588f43f6998e4c588f53f666/logs
{"error":"Status code '405 Method Not Allowed' was returned from log store","code":"5c81a8b7b2fc5b840a61f6e175e1d30a"}

[markusthoemmes]

You're absolutely right, it's confusing indeed. I think you'll need to append the _search in your configuration for it to work as of today.

To fix it properly, the client should append _search as its not part of the index.

Moreover, we should not return the status code of the logstore to the user like we do here. It should rather be a 502 and hide the details for the user.

cc @dubee

[rabbah]

@dubee is this something you can fix?

[dubee]

I'll have a look.