akka / alpakka

Alpakka is a Reactive Enterprise Integration library for Java and Scala, based on Reactive Streams and Akka.
https://doc.akka.io/libraries/alpakka/current/
Other
1.26k stars 645 forks source link

Does the elasticsearch connector support aggregations? #744

Open andys8 opened 6 years ago

andys8 commented 6 years ago

Is there a way to use alpakka and elasticsearch aggegrations (e.g. to calculate the sum)?

This example describes the query:

POST /sales/_search?size=0
{
    "query" : {
        "constant_score" : {
            "filter" : {
                "match" : { "type" : "hat" }
            }
        }
    },
    "aggs" : {
        "hat_prices" : { "sum" : { "field" : "price" } }
    }
}

ElasticsearchSource has only a few parameters (index, type, query and some configurations). But it looks like there is no way to provide the "aggs". Am I missing something?

Looking at the tests the query is always a match all query.

It looks like the connector is limited in the way the api is designed. I'm glad for any help. Thanks a lot.

ennru commented 6 years ago

You're right. For the time being the Elasticseach connector offers access to just a tiny bit of the Elasticsearch API. See #591

mbknor commented 6 years ago

I guess this is now possible with latest changes merged to master (not yet released):

https://github.com/akka/alpakka/commit/97f7e46fe41340e147115bb2680eb270150ac609#diff-a05dc5ecaacea54fb635b007c4260294R37

arunma commented 6 years ago

@mbknor @ennru Was browsing for issues for my first contribution and thought I could add some testcases for this issue. I've found out that unfortunately, the "aggs" query does not work even with the searchParams. Thought I could take a stab at it and ask for a review but unfortunately, I am stuck.

I've raised a Throw-away PR for your review. Appreciate if you could give some hints and help me move forward. Please ignore and close the PR if you have already started working on this issue. My apologies - I should have given a heads-up.

Link to the PR : https://github.com/akka/alpakka/pull/785

ennru commented 6 years ago

Alpakka does not support ES aggregations right now, as @arunma tried out. The aggregations reply data would need to be put into the stream in a good way, which is not simple in the current implementation.