Query Spike

[rbharath]

There are a number of high level questions we should start answering about queries. This issue lays out a first spike to start getting some query support into computable backends:

1. What does a query look like?
   • For now, this will probably just be a subset of SQL, but we need more detail on the specific subset. Is this the subset supported by Athena?
2. How can we get a query from the user to the backend system?
   • For example, is this through an ORM? Or even simpler, does the user send a raw SQL file over? Is there a REST endpoint the user sends the file to?
3. How do we validate the query?
   • Are there standard query sanitization tools? We should likely use Athena or SQLlite immutable or a similar system to make sure query isn't dangerous.
4. How do we get Flex set-up?
   • https://github.com/uber/sql-differential-privacy is Uber's differential privacy tool, Flex, for SQL. How can we get this up and running on the backend?
5. How can we compute privacy parameter epsilon for a given query?
   • Specifically, how do we run Flex on a given query
6. How do we send results back to the user.

[robrobbins]

subset of above: once a query is recieved and processed. what role does the data attribute of listings on the blockchain play in fetching the data pertinent to the query? There are, of course, the typical blockchain limitations on speed and the possible high gas cost of iterating a large list.

[geoffkinder]

@robrobbins, in my current mental model, I think listing.data is only relevant in associating a user address with a data set for the purposes of the data set being added or removed from a data marketplace. I don't believe a query needs to interact with the blockchain in any way. Since the data set has already been added to the data marketplace when a query is being submitted, I believe we can simply execute the query against our data store once it has been validated.

[rbharath]

This discussion is very outdated so closing.