search

datafuselabs / databend

𝗗𝗮𝘁𝗮, 𝗔𝗻𝗮𝗹𝘆𝘁𝗶𝗰𝘀 & 𝗔𝗜. Modern alternative to Snowflake. Cost-effective and simple for massive-scale analytics. https://databend.com
https://docs.databend.com
Other
7.41k stars 713 forks source link

help how to read a csv with variable number of columns #15861

Open djouallah opened 1 week ago

djouallah commented 1 week ago

I am doing a benchmarks to test the ETL performance for multiple Python Engine, I could not figure out how to do the same with databend, any help will be appreciate it

https://github.com/djouallah/Light_ETL_Challenge

youngsofun commented 1 week ago

copy into your_table_with_3_colums from (select $1, $2, $3 from @your_stage/path_to_csvs) file_format = (type=csv)

djouallah commented 1 week ago

csv has variable number of columns