Unable to Search Remote node

[TomPridham]

is there a way to enable verbose errors? i am having trouble querying a remote elasticsearch cluster, but the errors i am getting don't reveal anything that i could look into as a potential problem.

[mwilliammyers]

not really other than running with RUST_LOG=debug. I have been meaning to improve the error handling experience—particularly the default some API error occurred kind of message.

What errors are you running into?

[TomPridham]

that's the kind of error im running into lol. this is the error im getting: e=error sending a request or receiving a response. Caused by: error receiving a response. Status code: 400 Bad Request. the cluster is completely open and im able to get responses just by hitting the urls in the browser and the same code works with an elastic cluster running locally in docker. so im not sure what im missing. im using elastic@0.21.0-pre5 and an elasticsearch@7.4.2 cluster hosted in gcp. creating a client and using it like:

let es_url = env::var("ES_URL").expect("ES_URL must be set");
let client = AsyncClientBuilder::new()
  .static_node(es_url.clone())
  .serde_pool(Arc::new(ThreadPool::new()))
  .build();

...

client
  .document()
  .search()
  .index("media")
  .body(json!({})
  .send()

[mwilliammyers]

Yep, that is the generic error that we need to do something about... maybe logging the error is enough? Or I can see if I can make a serde_json::Value based error when we cant parse the error into a more Rusty type.

Hmm, try these in order:

1. Change to: .document::<serde_json::Value>()—off the top of my head, I am not sure if there is a default doc if you don't provide one (e.g.serde_json::Value)
2. I need to look at the document client code again to see if there is an issue for a search request with doc types which were removed in 7.0.0 (we are still working on updating to 7), but this should work:

   let res = client.search::<serde_json::Value>()
                   .index("_all")
                   .body(json!({ "query": { "match_all": {} } }))
                   .send()?;

3. Just for kicks, try (with your original query): json!({ "query": { "match_all": {} } }) for the query body.
4. Have you tried it without the serde_pool? I highly doubt that is the issue...

[TomPridham]

1: using serde_json::Value results in this compiler error. it compiles if i use the type i am expecting later(e.g. .document::<OptionMedia>, where OptionMedia is a struct i have defined elsewhere), but gives the same error from earlier.

             .search()
   |          ^^^^^^ the trait `elastic::types::document::impls::DocumentType` is not implemented for `serde_json::value::Value`

2: work and returns an empty object 3: works and returns an empty object if i delete the .document() so it seems like the issue is with .document(). i have no idea why it would work fine when running against my local ES cluster vs this remote one except that the shapes are prolly different. removing that line works for me, so im fine closing this unless you want me to try some other things to help you narrow down where the error is

thanks for your help. this has been driving me nuts

[mwilliammyers]

Ok, glad you got it working, I will leave it open to remind me to dig into the issue with .document().search()...

• What version is your local ES cluster?
• Do you expect it to return an empty object? Do you mean it doesn't return any documents i.e. [] (well really an iterator without any elements...)

[TomPridham]

the local cluster is 7.2. sorry, it was returning an empty object to postman from the endpoint i was hitting because there were no hits. the query comes back with this. which isn't what im expecting since the same query executed with curl brings back a bunch of results

{ took: 6, timed_out: false, shards: Shards { total: 1, successful: 1, failed: 0 }, hits: HitsWrapper { total: HitsTotal { value: 0, relation: Some("eq") }, max_score: None, inner: [] }, aggregations: None, status: None }

[TomPridham]

well, it turns out the empty results were from including a trailing slash in the connection url. myEsCluster.com/ vs myEsCluster.com, so all the requests were ending up with an extra slash. so, everything is working as expected now except for the .document() thing

[mwilliammyers]

Ahhhh! That has bit me before. Now that I know I’m not crazy, I’ll push a fix for that sometime this week.

And I’ll look at the .document() thing too.