The goal is to work with really large datasets and extract the results of large queries into a spark dataframe, this will allow us to work with pqs and spark to do large scale feature transformation and processing, we'll need to do a little design around this before implementation
The goal is to work with really large datasets and extract the results of large queries into a spark dataframe, this will allow us to work with pqs and spark to do large scale feature transformation and processing, we'll need to do a little design around this before implementation