search

NextCenturyCorporation / neon

http://neonframework.org
Apache License 2.0
55 stars 15 forks source link

Consider Apache Gora for as a storage abstraction layer for Data Access API #3

Closed lewismc closed 10 years ago

lewismc commented 10 years ago

Hi Folks, I am currently reviewing Neon as part of the Jet Propulsion Lab's Peer Review duties. I am project chair on Apache Gora [0] which provides an in-memory data model and storage abstraction API for Big Data. I see you guys currenty support MongoDB and Shark... Gora supports many datastores which could be made configurable within the Gradle build of Neon. If you guys are interested in pursuing this let me know and we can maybe improve Neon if this is something you would like to do. Thanks Lewis

[0] http://gora.apache.org

tpanningnextcen commented 10 years ago

Apache Gora looks very interesting, and the additional database support would be welcome to Neon. But Neon is driven by filtering and at this time, Gora’s filtering support doesn’t seem to be robust enough. Neon needs comparison filters (e.g., less-than, greater-than, etc.) that are processed by the database and properly handle the data types (numeric vs. string vs. date/time). Also, Neon needs aggregation (i.e., SQL's "group by") with support for things like sums, counts, and averages. Admittedly, for the use case that Gora is meant for, these capabilities aren't as important. We’re certainly open to reevaluating this if Gora gains these capabilities, or if they're already there and I missed them.

lewismc commented 8 years ago

Hi @tpanningnextcen we've improved the Filter API's in Gora and I wanted to update here with what we've been up to. Our FilterOp's include EQUALS, NOT_EQUALS, LESS, LESS_OR_EQUAL, GREATER and GREATER_OR_EQUAL. We also have a FiltrList API which enables MUST_PASS_ALL (AND) and MUST_PASS_ONE (OR) queries. We still don't have "group by" by we do have full data type support as these mappings are done on a datastore-by-datastore basis. If expanding datastore support for Neon is of interest then please get in touch with me. I would be very happy to help.