awslabs / python-deequ

Python API for Deequ
Apache License 2.0
713 stars 134 forks source link

Get full list of columns in check column of VerificationResult.checkResultsAsDataFrame. #181

Open gracemiguel opened 9 months ago

gracemiguel commented 9 months ago

Is your feature request related to a problem? Please describe. Hello, is there a way to get a full list of columns in the constraint column of the checkResultsAsDataFrame dataframe? I am trying to join the VerificationResult.checkResultsAsDataFrame with VerificationResult.successMetricsAsDataFrame but it is impossible unless I can get a full list of the columns used for each constraint in checkResultsAsDataFrame. If there is a multicolumn constraint, the check value is Stream(column1, ?) so I can't see all the columns. This is very frustrating because I need all the values from both dataframes to give the best results on my project. I have attached a photo showing the stream example.

Describe the solution you'd like

The most ideal solution would be a dataframe with the all the columns from both successMetrics and checkResults. If that can't be done, if you could allow a full list of column names in the constraint column instead of stream, I think I can manipulate it to join.

image

.