search

dinedal / textql

Execute SQL against structured text like CSV or TSV
MIT License
9.05k stars 300 forks source link

What problem are you solving? #71

Closed yanOnGithub closed 6 years ago

yanOnGithub commented 6 years ago

I have read #62 and https://github.com/dinedal/textql#key-differences-between-textql-and-sqlite-importing. How about this shell script?

image

dinedal commented 6 years ago

Hi, thanks for checking out TextQL.

If you have an issue with the software, please feel free to open a new issue. I don't believe this is a bug report, but instead I guess you're trying to ask a question "what's the point of TextQL?"

The point is that I wrote it to solve a problem I had; I wanted a fast way to access a text file using SQL statements, without having to create a database in Postgres/MySQL. I also had issues using Excel or Open Office via the command line, and couldn't push huge files to a cloud service for processing. That's it. Every feature in TextQL I added because I ran into a real problem using text delimited files and needed to solve it. I opened sourced TextQL because I felt that these were common problems. It turns out, many people have found TextQL useful because those same problems have bothered them as well.

If your delimited file has mis-aligned rows, TextQL will read it. It tries to be liberal in what it accepts and strict in it's output. If your delimited file is many GB in size, TextQL will use an in-memory database automatically to speed it up. Even if it can't, it always loads it in the most efficient way possible. If you want to join multiple text files based on a column, TextQL will do it. If you don't want to type a ton, TextQL supports short-hand syntax for SQL, automatically naming tables and inserting table names into queries. If you want to stream data in from stdin, TextQL allows you do it, and also to stdout.

There are many other competing projects out there, including the one which you just wrote. There are others in Perl, Python, etc. Some have more features, some have less. Unrelated; your code here is a screenshot, might I suggest you make it open source?

The other projects all vary in features being offered. None of them; as far as I know, offer what TextQL does. I wanted those features, so I added them to TextQL. If you don't want/need them, feel free to use your software, or any other software.

Thanks again for looking at TextQL.