antonycourtney / tad

A desktop application for viewing and analyzing tabular data
http://tadviewer.com
MIT License
3.19k stars 119 forks source link

use list parameter to read 3 parquet files and treat them as a single table #200

Open AFgh24 opened 1 year ago

AFgh24 commented 1 year ago

Please add this feature

To search among multiple files Each file must be opened one by one for each search which is very time consuming While there is a function command in duckdb for this Thanks

-- use list parameter to read 3 parquet files and treat them as a single table

SELECT * FROM read_parquet(['file1.parquet', 'file2.parquet', 'file3.parquet']); -- read all files that match the glob pattern

SELECT FROM 'test/.parquet'; -- read all files that match the glob pattern, and include a "filename" column that specifies which file each row came from SELECT FROM read_parquet('test/.parquet', filename=true);

antonycourtney commented 1 year ago

While not exactly what you are asking for, if you run tad from the command line, you can pass the path to a directory instead of just a single file, and use the data source browser to quickly switch between different files.

You can see an example of this in the animated GIF on the Tad guide on the DuckDb page here: https://duckdb.org/docs/guides/data_viewers/tad

Hope this helps!

AFgh24 commented 1 year ago

hi Is it possible to add the ability to search multiple files in the next version? Search for specific text in multiple parquet files

This will increase the search speed

Open the files in a folder one by one and search for the desired text. After finishing the file, go to the next file until the files are finished.

By doing this, you are doing me a great favor (I hope you will add this feature, it is very useful)

Thanks