search

multiprocessio / datastation

App to easily query, script, and visualize data from every database, file, and API.
https://datastation.multiprocess.io
Other
2.9k stars 112 forks source link

[DSQ] Passing multiple files as different "tables" and doing joins #132

Closed TheKnarf closed 2 years ago

TheKnarf commented 2 years ago

For the DSQ tool:

Seems like this tool would be a lot more useful if I could pass multiple files, naming each of them as a "table" name, and then doing joins on the data. Would be especially useful if one could do this with different datatype, maybe I have a couple of Excel documents, a csv and some json data coming from different datasources and I'd like to run a query over all of them.

Current workflows involves writing data importers to get all of the data into a proper SQL database server, and then running queries against to generate a report.

eatonphil commented 2 years ago

Thanks for the suggestion! Yes this should be doable just by extending the existing filename lookup/read logic.

eatonphil commented 2 years ago

Hey @TheKnarf this is now available in dsq. dsq now has its own repo at https://github.com/multiprocessio/dsq. Feel free to follow the install instructions and README for multiple files there and give it a whirl.

Thanks again for your feedback!

TheKnarf commented 2 years ago

Cool! Love that it got its own repo. Just starred the new repo. Great job!