elastic / eland

Python Client and Toolkit for DataFrames, Big Data, Machine Learning and ETL in Elasticsearch
https://eland.readthedocs.io
Apache License 2.0
627 stars 98 forks source link

Idea: Adding the possibility to create a Dataframe from a KQL query #684

Open bartbroere opened 3 months ago

bartbroere commented 3 months ago

This PR is a first draft of how we could create a DataFrame by filtering an index with Kibana Query Language.

I found the kql module elsewhere in the Elastic Github organisation. For now I submodule+symlinked it in here, but there probably is a better way to do that.

You would use this by doing something like this:

import eland

eland.Dataframe('http://localhost:9200', 'example-index').kql_query('field1 > 100 and (field2:"value1" or field2:"value2")')
bartbroere commented 3 months ago

I'll be mostly offline during April, so this is just a heads up that this PR will not see much progress for a few weeks. I didn't forget about it though, and plan to continue working on it in May.