Error using com.amazon.deequ.analyzers.runners.MetricCalculationRuntimeException: org.apache.spark.sql.AnalysisException: cannot resolve '`Data.Id`' given input columns

awslabs / python-deequ

Python API for Deequ

Apache License 2.0

713 stars 134 forks source link

Error using com.amazon.deequ.analyzers.runners.MetricCalculationRuntimeException: org.apache.spark.sql.AnalysisException: cannot resolve '`Data.Id`' given input columns #77

Open laurinhocode opened 3 years ago

laurinhocode commented 3 years ago

Describe the bug When i pass the df to ColumnProfilerRunner and the column has a dot e.g: Data.Id the following error appears:

"Error using com.amazon.deequ.analyzers.runners.MetricCalculationRuntimeException: "org.apache.spark.sql.AnalysisException: cannot resolve 'Data.Id' given input columns"

The call: result = ColumnProfilerRunner(spark) \ .onData(df) \ .run()