Open jgbarah opened 6 years ago
When I implemented areas of code and onion studies I tried to use a simple approach for reading and writing: https://github.com/chaoss/grimoirelab-elk/blob/master/grimoire_elk/enriched/ceres_base.py#L36
Then you can use a common code to read: https://github.com/chaoss/grimoirelab-elk/blob/master/grimoire_elk/enriched/ceres_base.py#L90
and write: https://github.com/chaoss/grimoirelab-elk/blob/master/grimoire_elk/enriched/ceres_base.py#L96
I think we should follow an approach similar to this one (maybe not so simple, or maybe even simpler) in such a way that producing new indexes should be as easy as implement the data access layer and use it together the code to process data.
Thanks for the links to Ceres, @alpgarcia. Yes, I was thinking about something like that, but I don't know how difficult it would be to implement that in the current GrimoireELK codebase...
Now, GrimoireELK is targeted at producing raw and enriched indexes in Elasticsearch. But in some scenarios, getting a CSV file with the information we usually have in an enriched index would be very convenient, specially if that can be done without accessing Elasticsearch.
Reading the code, it seems this is maybe not straightforward, but not that difficult either. And maybe this could be a first step in decoupling GrimoireELK from Elasticsearch.
Please, feel free to add comments now the idea, including to which extent it makes sense to you, and you would implement it.