[FEA] Fingerprint tables by reading the statistics from external file

[nartal1]

Is your feature request related to a problem? Please describe. This issue is to add ability to create representative datasets based on statistics of input datasets that are part of join operation.
This issue is create a prototype and to track exploratory work to get the statistics that is required to generate representative data for joins and HashAggregates(if there are spills) Reference issue: https://github.com/NVIDIA/spark-rapids/issues/11239

• [ ] Get the statistics from the input datasource(tables) and save it to output file. Start with parquet file and numeric types.
• [ ] Generate representative data based on the information from the above step.
• [ ] From the eventlog get the join type, buildside(if applicable), number of input rows, number of output rows after the join operation - This is out of scope for the initial POC.
• [ ] Investigate the statistics that are important if there are spills in HashAggregate operation - This is out of scope for initial POC.

[amahussein]

Currently, I am working on a few issues related to datasourceInfo like #1172 I believe this would change the current implementation of extracting datasources and the details of SQLInfo

[nartal1]

Currently, I am working on a few issues related to datasourceInfo like #1172 I believe this would change the current implementation of extracting datasources and the details of SQLInfo

Thanks @amahussein. I will discuss with you. Currently I am looking into getting the stats from tables(assuming we have table name).