Closed BhuviTheDataGuy closed 4 years ago
Hi Bhuvi,
Spectrify uses the Apache Arrow project to write Parquet files. Behind the scenes, Arrow uses the Apache-managed C++ parquet writer, parquet-cpp.
More info here: https://arrow.apache.org/docs/python/parquet.html
Hey, this tool is amazing and simplified data engineer's life.
Im trying to understand the principles of this tool.
It's converting the CSV to Parquet, just curious how it's doing this process without any Hadoop clusters?