As use of CSVs as datasources makes its way into edges, we can see there's a disconnect between how all the data sources are used. Components basically need a lot of knowledge about where data is stored and what format it is in, which makes them highly customised to their position. It would be nice to have a standard filtering/aggregating/results interface behind which both es and csvs sit. We would then be able to add other data sources more easily, such as Solr, Lunr, XML, JSON, etc.
Not sure what the right format for that interface is, but I'd probably begin with the ES query and results syntax, and force the other data sources to conform to that via an adaptor.
As use of CSVs as datasources makes its way into edges, we can see there's a disconnect between how all the data sources are used. Components basically need a lot of knowledge about where data is stored and what format it is in, which makes them highly customised to their position. It would be nice to have a standard filtering/aggregating/results interface behind which both es and csvs sit. We would then be able to add other data sources more easily, such as Solr, Lunr, XML, JSON, etc.
Not sure what the right format for that interface is, but I'd probably begin with the ES query and results syntax, and force the other data sources to conform to that via an adaptor.