ElasticSearch query doesn't apply

n8n-io / n8n

Free and source-available fair-code licensed workflow automation tool. Easily automate tasks across different services.

https://n8n.io

Other

47.38k stars 7.1k forks source link

ElasticSearch query doesn't apply #11258

Open igorvolocRC opened 6 days ago

igorvolocRC commented 6 days ago

Bug Description

When executing a search query in the Elasticsearch node, the intended filtering is not applied. Instead of returning the correct subset of matching documents, the search retrieves all the most recent documents up to the specified limit or the maximum default limit of 10,000 documents.

I noticed this after the upgrade from 1.55.2 -> 1.62.5

Tomorrow, I'll provide more details.

To Reproduce

Write any query to filter some documents for an index
when executing the node, the query will not be applied

Expected behavior

The query should be applied to filter documents.

Operating System

n8nio/n8n:1.62.5

n8n Version

1.62.5

Node.js Version

n8nio/n8n:1.62.5

Database

PostgreSQL

Execution mode

main (default)

Joffcom commented 6 days ago

Hey @igorvolocRC,

We have created an internal ticket to look into this which we will be tracking as "GHC-317"

Joffcom commented 6 days ago

Hey @igorvolocRC,

Are you sure this worked in 1.55.2? I can't see any changes that would have impacted this option, I will see if I am able to reproduce this later today though.

igorvolocRC commented 5 days ago

This is the result of a quick test. The left side is 1.55.2, and the right side is 1.62.5. The workflow and credentials are the same. As you can see, the number of returned items is different for the elasticsearch node. In the 1.62.5, the result is 600 because I set the limit; otherwise, it will take some time until it will return the hardcoded 10.000. It's definitely an issue with the elasticsearch node because the same API request with the "HTTP Request" node returns the correct number of documents in both versions.

I'll do more research, but now I have to think about downgrading because the DB has some migrations from 1.62.5 and will not start with 1.55.2.

Left side image

n8nVersion: 1.55.2
platform: docker (self-hosted)
nodeJsVersion: 20.15.0
database: sqlite
executionMode: regular
concurrency: -1
license: community
consumerId: unknown

Right side image

n8nVersion: 1.62.5
platform: docker (self-hosted)
nodeJsVersion: 20.18.0
database: sqlite
executionMode: regular
concurrency: -1
license: community
consumerId: unknown

Joffcom commented 4 days ago

That is interesting, thanks for testing. I will dig through the release notes and see if I can spot the change.