We should have a full Structure to import the data from the pipeline directly to any other file in the repo. So we should have this as a pipeline with all the steps included.
I will make a vanilla pipeline shortly, but @0ardneebwar0 could you start working on this(start with a notebook and we will integrate it later in the pipeline)?
also, just put a comment with the columns it will include initially, so others can take this into consideration simultaneously.
We should have a full Structure to import the data from the pipeline directly to any other file in the repo. So we should have this as a pipeline with all the steps included.
I will make a vanilla pipeline shortly, but @0ardneebwar0 could you start working on this(start with a notebook and we will integrate it later in the pipeline)?
also, just put a comment with the columns it will include initially, so others can take this into consideration simultaneously.