A connector in charge of persisting context data sources into other third-party databases and storage systems, creating a historical view of the context
Apache Parquet is a columnar storage format available to any project in the Hadoop ecosystem, regardless of the choice of data processing framework, data model or programming language. (http://parquet.incubator.apache.org/)
Parquet is a format some times requested by FIWARE users as a useful way of persisting historical Orion context data in HDFS.
Adding this new format will imply the extension of the available values about the file_format configuration parameter in NGSIHDFSSink (currently, json-row, json-column, csv-row and csv-column. We will be adding parquet-row and parquet-column, if such a distiction is possible, of simply parquet.
Apache Parquet is a columnar storage format available to any project in the Hadoop ecosystem, regardless of the choice of data processing framework, data model or programming language. (http://parquet.incubator.apache.org/)
Parquet is a format some times requested by FIWARE users as a useful way of persisting historical Orion context data in HDFS.
Adding this new format will imply the extension of the available values about the
file_format
configuration parameter inNGSIHDFSSink
(currently,json-row
,json-column
,csv-row
andcsv-column
. We will be addingparquet-row
andparquet-column
, if such a distiction is possible, of simplyparquet
.