search

stefan-hoeck / idris2-sqlite3

Idris2 bindings to the sqlite3 C-API
BSD 3-Clause "New" or "Revised" License
3 stars 1 forks source link

Add a roadmap #37

Open emdash opened 7 months ago

emdash commented 7 months ago

I feel like this library off to a good start, but this project could use a roadmap to help prioritize the efforts of contributors.

stefan-hoeck commented 7 months ago

I'm open to suggestions here. I have to admit that I'm no SQL expert, so I don't have a clear picture what important features are still missing. Typically, I just implement the stuff that comes up as necessary in my own projects.

emdash commented 7 months ago

Neither am I. What a pair we are :)

I've basically been exploring the railroad diagrams that are documented here:

To anchor the discussion, I found these crazy SQL examples in the WITH statement documentation: https://www.sqlite.org/lang_with.html#outlandish_recursive_query_examples . One example is a query that outputs a mandlebrot set, and the other solves sudoku. This is alongside queries that traverse graphs and produce tree expansions. Supporting queries like this represents one "endpoint" in the development of the library.

You managed to automate some of the DOM binding library, do you have any insight to share here?

I feel like reading the sqlite3 source might be worthwhile. I learned from an episode of corecursive (I think) that sqlite3 processes queries as follows:

So perhaps by studying the parser, compiler, and vm instruction set, one could perhaps extract some intuition for how to model the language. And from there, perhaps find a path to cover most of SQL "in one go", with the help of some meta-programming magic.

On the other hand, one could analyze some corpus of SQL and try to find some minimum viable subset of the language, and target less-automated development efforts on that.

For my part, the former approach seems more appealing. Perhaps someone I know is well-versed on SQL, and can share their knowledge -- I will ask around.

emdash commented 7 months ago

I did spend some time looking in the SQL codebase. It's very much a C project. But I was able to explore the SQL parser implementation, and the VM instruction set.

The good news is that the VM instruction set seems manageable. The bad news is that the grammar file for the parser is fairly complex, and the code which processes the resulting AST is also relatively obtuse. So, studying the implementation might be more time-consuming than I had originally thought, but I will give it some time before I abandon the idea.

I can post a more detailed summary of what I've found so far, if it interests you. My goal, I guess, would be to come up with some intuition for how SQL queries get compiled to bytecode, and a similar intuition for how the VM interprets bytecode.

stefan-hoeck commented 7 months ago

I did spend some time looking in the SQL codebase. It's very much a C project. But I was able to explore the SQL parser implementation, and the VM instruction set.

The good news is that the VM instruction set seems manageable. The bad news is that the grammar file for the parser is fairly complex, and the code which processes the resulting AST is also relatively obtuse. So, studying the implementation might be more time-consuming than I had originally thought, but I will give it some time before I abandon the idea.

I can post a more detailed summary of what I've found so far, if it interests you. My goal, I guess, would be to come up with some intuition for how SQL queries get compiled to bytecode, and a similar intuition for how the VM interprets bytecode.

Thanks for looking into this!