CrossClusterSearch - Can't get field metadata

[pyro2927]

Elasticsearch version (bin/elasticsearch --version): 6.5.2

Plugins installed: []

JVM version (java -version):

[ec2-user@XXXXXXX ~]$ java -version
openjdk version "1.8.0_191"
OpenJDK Runtime Environment (build 1.8.0_191-b12)
OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)

OS version (uname -a if on a Unix-like system):

[ec2-user@XXXXXXX ~]$ uname -a
Linux ip-10-92-51-187 4.14.77-70.59.amzn1.x86_64 #1 SMP Mon Nov 12 22:02:45 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Description of the problem including expected versus actual behavior:

CrossClusterSearch doesn't allow for anything that requires a little information before the search is executed. Specifically, it does not work in Kibana's Canvas, nor as a verified data source in Grafana. The workaround in Grafana is to save the data source and ignore the error of the test passing (graphs will load and work), however there is no workaround for Kibana.

I believe this is due to the fact that field mappings and metadata are unable to be fetched from the remote cluster. These steps are executed in order to validate the index, and cause the corresponding clients to stop attempting execution.

Screenshot of error from Grafana (though it still works as a data source):

Screenshot from Canvas as it tries to convert ESSQL into a search query (does not work, no workaround):

This functionality would be useful because it would allow us to do federated search through a single ES Cluster into many other other managed clusters. This is something we are interested in doing through ECE to manage the clusters.

Steps to reproduce:

Please include a minimal but complete recreation of the problem, including (e.g.) index creation, mappings, settings, query etc. The easier you make for us to reproduce it, the more likely that somebody will take the time to look at it.

1. Set up two ElasticSearch clusters (A & B)
2. Create an index in B
3. Add B as a remote cluster in A's configuration
4. Perform a search of B:index/_search from A (works)
5. Attempt to get B:index/_mapping from A, fails

[elasticmachine]

Pinging @elastic/es-search

[jimczi]

The mapping API doesn't handle cross cluster search so the error is expected. Kibana uses another API called field_caps to retrieve field mapping, this API can handle cross cluster so you can use remote indices in your index patterns. SQL doesn't handle cross cluster yet but this is planned so we' should add the support soon. I don't know the plan for Canvas so you should probably open an issue in the Kibana repo to get some informations. Grafana is a separate project so I have no idea if and how it should support cross cluster search. I hope you don't mind if I close this issue, there is no bug or feature to add in Elasticsearch here.