search

thanos-io / promql-engine

Multi-threaded PromQL engine implementation based on the Volcano paper.
Apache License 2.0
142 stars 55 forks source link

LFX: Research querying Apache Parquet files #167

Open fpetkovski opened 1 year ago

fpetkovski commented 1 year ago

Since we have a separation between the logical plan and execution, we can allow querying arbitrary data sources using PromQL.

One such data source could be Apache Parquet files, similar to how https://github.com/polarsignals/frostdb currently works. We can also investigate options to directly integrate with FrostDB if that makes querying Parquet simpler.

Shubham4359 commented 1 year ago

Hey @fpetkovski I would like to work on this issue in the upcoming LFX mentorship term. I have previously worked with Apache Parquet files where i had to write given metadata to parquet files. Doing query on top of Parquet files would be the next stage of exploration for me. Can you give me a brief idea about the prerequisites needed to learn and is there anything else that needs to be looked into?

fpetkovski commented 1 year ago

Hey @Shubham4359, thanks for expressing interest in this project. Feel free to apply through the LFX website :)

Althaf66 commented 1 year ago

I am interest in this project.Can you give the information about the project breifly and its prerequesites.

fpetkovski commented 1 year ago

Here's a link to the LFX project: https://mentorship.lfx.linuxfoundation.org/project/a04cfbe4-4dde-4c7e-8b70-9570639b48a7.

Althaf66 commented 1 year ago

Hi @fpetkovski ,Thanks for the information.I like to work on this issue in the LFX mentorship term. I requested to give prerequisites and tools or techstack to complete this project.

fpetkovski commented 1 year ago

Hi @Althaf66, knowledge of golang should be sufficient for this project.

vishalvivekm commented 1 year ago

Hi @fpetkovski I am a beginner learning go. I am really interested to contribute to this project as a Linux Foundation mentee for the upcoming spring term. After applying, I was curious if I should focus solely on finding and contributing to issue or I should focus on learning about the go and project-requisites in more detail ? can you please help me on this!

fpetkovski commented 1 year ago

Hi @vishalvivekm, feel free to look at the issues with a good-first-issue tag :)

Sagar180304 commented 1 year ago

Hi @fpetkovski I am looking forward to contribute in these issue under LFX Mentorship 2023

vishalvivekm commented 1 year ago

Hi @vishalvivekm, feel free to look at the issues with a good-first-issue tag :)

Roger That. I'll be looking for them to make contributions. Thanks :)

Zaher1307 commented 1 year ago

Hi @fpetkovski, I will be glad to work in this issue in LFX mentorship. I have previous work experience in Golang and I have already applied in LFX website, so what should I have to do to contribute?

douglascamata commented 1 year ago

There has been an LFX mentee selected to work on this: @Shubham4359, as communicated through the #thanos-dev CNCF channel.

yeya24 commented 1 day ago

What's the next step of this issue? Do we plan to contribute another storage scanner implementation https://github.com/thanos-io/promql-engine/blob/main/storage/interface.go or we can close this issue?