mechatroner / RBQL

🦜RBQL - Rainbow Query Language: SQL-like query engine for (not only) CSV file processing. Supports SQL queries with Python and JavaScript expressions.
https://rbql.org
MIT License
281 stars 13 forks source link

Add support for multiple tables #33

Open seppestas opened 2 years ago

seppestas commented 2 years ago

It would be great if this tool could have support for more than one or two tables. Some ideas to make this happen:

I'm currently using CSV virtual tables in SQLite and Numpy to achieve things like this, but rbql is a lot easier to get up and running and the availability of both a Python and JS library is very appealing. Runtime performance might become a concern though.

seppestas commented 2 years ago

A more elaborate example of what I would really love would be possible:

Filesystem:

devices.csv
testresults/
    rx-results-20220107.csv
    tx-results-20220107.csv
rbql --query SELECT device.id, device.name, rx.rssi, tx.tx_power FROM devices.csv AS device JOIN testresults/rx-results-20220107.csv AS rx ON device.id == rx.device_id JOIN testresults/tx-results-20220107.csv AS tx ON device.id == tx.device_id
mechatroner commented 2 years ago

I think this sounds like a good roadmap, thanks for your ideas! I actually was already thinking about adding FROM keyword since I need this for ipython magic %rbql command that should be able to work with pandas data frames. Regarding multi-table joins - this is one of the directions for further development, but each feature makes adding other features and maintenance exponentially harder as I already discovered, so right now I am really careful about things like this. Because having good UI/UX, performance and other things are also important. BTW it is actually possible to emulate multi-table joins with consecutive 2-way joins, although it is not very convenient. An option to find csv tables (recursively?) and display them is nice and it is possible to do this outside of the RBQL engine, so this is an interesting option, but to really utilize this RBQL needs some GUI clients like a text UI client for command line and maybe a real browser/electron-based GUI.