search

elastic / gsoc

Google Summer of Code at Elastic: Projects and guidelines
44 stars 12 forks source link

GSOC18: Web Interface to Index Data To Elasticsearch #18

Closed aravindiiitb closed 6 years ago

aravindiiitb commented 6 years ago

Hi Team,

Hope you all are doing good. I have used Elasticsearch for my academic projects.

Right now, users write python scripts to index the data to their Elasticsearch clusters and then add that index in the Kibana portal.

If a thing is being repeated multiple times it needs to be automated right ? What I would like to suggest here is, I will create an web UI , where user can upload a csv or point to a data source or write an sql query (to fetch the data), and user will also provide the index name which he want to create.

That's it , That is all what I need, all the hard work to index the data to Elasticsearch cluster will be handled in the background. This is very much do-able if I have your support.

Keeping in mind the number of users who use the Elasticsearch , this web UI will save a lot of time, which user is spending to write scripts to index data to Elasticsearch now.

I hope I've conveyed the idea properly. Let me know if something is not clear in regarding to this proposal.

Looking forward to hear from you soon.

Thanks & Regards, Aravind Kashyap aravindiiitb

xeraa commented 6 years ago

Closing the loop here, since we've also discussed this by mail:

Thanks for coming up with your own project idea — we appreciate people taking some initivative!

We almost had this feature in Kibana actually. You can still see a description and screenshot in the 5.0.0-alpha 4 announcement: https://www.elastic.co/blog/kibana-5-0-0-alpha4 Though that never made it into a stable release, because mappings and date formats where always tricky with a lot of edge cases. Also if you made a mistake in your import you had to delete the index and start over. Our assumption was that CSV ingestion in Kibana is a tool mainly for new users and we couldn’t reach an experience that was smooth enough for all possible scenarios and use cases.

We had a quick discussion internally and the conclusion was that we don’t think that this would be a great GSoC project, because there is a very real chance that it would fail (again). We really want our students to have a good experience, so unfortunately this won’t have a good chance of being accepted.

That being said, this was a great idea and has been requested multiple times, so if you have any other project ideas not mentioned on our page yet, don't be shy and open another issue to discuss them :)